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(Manuscript received July 18, 1960) 


Theoretical questions concerning the possibilities of proving theorems by 
machines are considered here from the viewpoint that emphasizes the under- 
lying logic. A proof procedure for the predicate calculus is given that con- 
tains a few minor peculiar features. A fairly extensive discussion of the 
decision problem is given, including a partial solution of the (x)(Ey)(z) 
satisfiability case, an alternative procedure for the (x)(y)(£z) case, and a 
rather detailed treatment of Skolem’s case. In connection with the (x)(Ey)(z) 
case, an amusing combinatorial problem is suggested in Section 4.1. Some 
stmple mathematical examples are considered in Section VI. 

Editor’s Note. This is in form the second and coneluding part of this paper’ 


Part [ having appeared in another journal.! However, an expansion of the author’s 
original plan for Part II has made it a complete paper in its own right. 


I. A SURVEY OF THE DECISION .PROBLEM 


1.1 The Decision Problem and the Reduction Problem 


With regard to any formula of the predicate calculus, we are interested 
in knowing whether it is a theorem (the problem of provability), or 
equivalently, whether its negation has any model at all (the problem of 
satisfiability). Originally this decision problem was directed to the search 
for one finite procedure which is applicable to all formulae of the predi- 
cate calculus. Since it is known that there can be no such omnipotent 


l 
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procedure, the main problem is to devise procedures effective for classes 
of formulae which satisfy suitable conditions. 

The complementary problem of reduction is to give effective proce- 
dures which reduce broader classes to narrower ones while preserving 
provability or satisfiability. In this way, a decision procedure for a 
smaller class can be made to apply to a larger one. Thus far, most work 
on the reduction problem has been directed to the special case of finding 
procedures which reduce all formulae of the predicate calculus to mem- 
bers of some special class (e.g., those in the Skolem normal form). Each 
such class is called a reduction class relative to satisfiability or provabil- 
ity according to whether satisfiability or provability is preserved by the 
transformations (Ref. 2, p. 32). It follows automatically that the corre- 
spouding decision problem for each reduction class is unsolvable. 

The reduction classes and the procedures employed to obtain them 
are, being concerned with undecidable cases, only of indirect use for the 
problem of discovering positive results on the decision problem. More 
directly relevant are reduction procedures which are applicable when the 
reduced class is not a reduction class and may in particular be a decid- 
able class. Some very preliminary results on this more general aspect 
of the reduction problem will be described in Section V. 

For both the decision problem and the reduction problem, there is, 
beyond the “‘yes or no” as to satisfiability, a further question of deter- 


mining all models and devising transformation procedures which preserve 


all models. Such questions have been studied to a certain extent (Ref. 
3, p. 23), but will be disregarded in what follows. 

It is customary to characterize reduction classes and decidable classes 
in terms of formulae in the prenex normal form, i.e., with all quantifiers 
at the beginning. Sometimes, with regard to satisfiability (or provabil- 
ity), conjunctions (or disjunctions) of formulae in the prenex normal 
form are considered. We shall call this the extended prenex form. 

In Section V, a procedure will be given for reducing any formula to 
a finite set of generally simpler formulae in the extended prenex form 
such that the original formula is provable if and only if all formulae in 
the reduced set are. In this and the next few sections, we shall only be 
concerned with formulae in the extended prenex form. Furthermore, 
we shall give in Section V a proof-decision procedure for the quantifier- 
free logic, obtained from the propositional calculus by adding equality, 
function symbols and individual constants. Any theorem in it is called 
a quantifier-free tautology, as an extension of the notion of a propositional 
tautology. We shall make use of the fact that we can always decide 
whether a given formula is a quantifier-free tautology. 
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1.2 A Brief Formulation of the Predicate Calculus 


1.2.1 Primitive Symbols 


1.2.1.1 Variables x, y, z, etc. (an infinite set). 

2.1.2 Individual constants (a finite or infinite set). 

2.1.3 Propositional (Boolean) operations: ~, v, &, D, =. 

2.1.4 Predicate letters (a finite or infinite set). 

.5 Function letters (a finite or infinite set). 

Equality: = (a special predicate symbol). 
Quantification symbols: ( ), (F ). 
Parentheses. 


1.2.2 Inductive Definition of Terms and Formulae 


1.2.2.1 A variable or an individual constant is a term. 

1.2.2.2 A function symbol followed by a suitable number of terms is 
a term. 

1.2.2.3 A predicate followed by a suitable number of terms is a for- 
mula (and an atomic formula); in particular, if a, 8 are terms = (a,@) 
or a = 8 isa formula (and an atomic formula). 

1.2.2.4 If ¢, w are formulae and a is a variable, then (a)g, (Ka)¢, 
~yevb,¢ &Vv,e Dy, ¢ = vare formulae. 


1.2.3 Inductive Definition of Theorems 
‘ 

.2.3.1 A quantifier-free tautology is a theorem. 

.2.3.2 If a disjunction D of n alternatives is a theorem, ¢a is one of 
the alternatives and @ is a variable, then: 

(a) If @ is a term, then the result of replacing ga by (£8)¢@ in D is 
a theorem; 

(b) if a is a variable free in ga but not free in the other alternatives 


and 6 is @ or does not occur in ga, then the result of replacing ga by 
(8)¢@ in D is a theorem. 
1.2.3.3 If ¢ v --- v gis theorem, so is also ¢. 


The above formulation is complete only with respect to formulae in 
the extended prenex form. 
1.3 The Fundamental Theorem of Logic 


The main purpose of the next few sections is to study the decision 
problem on the theoretical foundation of the fundamental theorem of 
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. . ae 1 4 5 
logic, an approach initiated by Skolem’ and Herbrand,’ and recently 

. ‘ 5,7 r § 9,10 
revived by Church,°” and by Klaua® and Dreben.’"" 

Suppose .Wryz is a quantifier-free matrix: 

(x) (Ey) (z)Mayz, 
(Ex)(y)(Ez) ~ Maye. 

Let now D, be M, v «++ v M, and M; be M11’, 7’ being an abbre- 

viation for 7 + 1. The fundamental theorem, when applied to 1.3.1, 


states: 


1.3.3 The following three conditions are equivalent: 
(a) 1.3.1 is a theorem of the predicate calculus; (b) for some n, D, is 
a quantifier-free tautology; (c) 1.3.2 is not satisfiable. 

If D, is a quantifier-free tautology, then, by 1.2.3.1, both it and the 
result of substituting distinct variables for distinct numbers in it are 
theorems. For example, suppose the result is: 


1.3.4 Maab v Mabe v Macd. 
We have: by 1.2.3.2(b), 
Maab v Mabe v (z)Macz; 
by 1.2.3.2(a), 
Maab v Mabe v (Ey)(z)Mayz. 
Similarly, 


Maab v (Ey)(z)Mayz v (Ey)(z)Mayz, 
(Ey)(z)Mayz v (Ey)(z)Mayz v (Ey)(z)Mayz, 


by ie © &, 4 
(Ey) (z)Mayz; 
by 1.2.3.2(b), 
(x)(Ey)(z)Mayz. 


Hence, condition (b) implies conditions (a) and (¢) in 1.3.3. 


On the other hand, if no D, is a quantifier-free tautology, then there 


is, for each D, , some interpretation of the function and predicate sym- 
bols on the set {1, --- , ’} which satisfies ~D,, . By a well-known argu- 
ment, there is then an interpretation on the domain of all positive in- 
tegers which satisfies ~/)),, ~ D2, ete. simultaneously. This, however, 
means that under the interpretation each finite segment of the infinite 
conjunction 
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1.3.5 ~M112 & ~M123 & ~M134 & --- 


is true. But then there is an integer x, viz. 1, such that for every integer 
y, there is an integer z, viz. y’, such that ~Mzyz. In other words, 1.3.2, 
the negation of 1.3.1, is true under the interpretation. Hence, the nega- 
tion of condition (b) implies the negations of conditions (a) and (ce). 

If we take 1.3.5 as a model of 1.3.2, it seems natural to regard y as 
an independent variable, z as a dependent variable and z as an initial 
variable (the limiting case of a dependent variable, a function of zero 
arguments). The general principle of constructing M, from 1.3.1 may 
be summarized by saying that each initial variable gets a constant 
number, the independent variables taking on all possible positive in- 
tegers as values and the dependent variables always taking on numbers 
not used before. 

In the general case, we must consider a disjunction (for provability) 
or conjunction (for satisfiability) of formulae with arbitrary strings of 
quantifiers. Then we can again construct the related quantifier-free 
formulae in the same way, with the numbers in each clause proceeding 
independently. 

Thus, if we wish to study the satisfiability problem, we consider any 
formula of the form: 


1.3.6 G1 Wess & Gn 


where each ¢; is of the form, with d, 
* Gy & I: 


1.3.7 (Hy") +++ (Bya,')(a') «++ (ae') +++ (Bin) --- 


(Eya, (ar) +++ (xe My! +++ x,°. 


¢ 


One familiar way of obtaining M,, M2, ete. for the formula 1.3.7 
begins by replacing the dependent variables (those with the letter y) 
ach with a function (sometimes called a “Skolem function’’) of all the 
preceding independent variables (those with the letter x), and then 
dropping all the quantifiers. Let the result be 1/*. In particular, the 
initial (dependent) variables are replaced by distinct constants which 
may be viewed as trivial functions. Suppose e; + --- +e = p, dy + 
ee +d, = qin 1.3.7. 

The Skolem functions are any functions g;, «-:, g, which, taken 
together, satisfy the following conditions: 


1.3.8 (a) For each g;, gilt, «++, Un) #F Uj, 
i - i. a q. 
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(b) For each g;, gilt, +--+, Um) = gilt, «++, Um) Only when wm = 


Vj, °°* > tm = Vm. 


(ec) For any g;, 9;, 7 #7, g(t, «++, Um) # gir, +++, Un), for all 


My, *** , Mas M1» > Un- 


Then we can take the smallest domain which contains the constants 
for the initial (dependent) variables (or an arbitrary constant when 
there is no such initial variable) and is closed with respect to the Skolem 
functions. Once such an (enumerable) domain is available, we can some- 
how enumerate all the p-tuples of members of the domain. Then, for 
each i, M; is simply the result obtained from .7* when the independent 
variables are replaced respectively by members of the ith p-tuple. 

The satisfiability problem of 1.3.7 is then reduced to that of the infinite 
conjunction : 


1.3.9 M,&M.&::: 


Similarly, the satisfiability problem of 1.3.6 can be handled by reducing 
each ¢; separately and then taking the conjunction of the n infinite 
conjunctions of the form 1.3.9. 

It is customary to use the positive integers as the domain, fix some 
enumeration of the p-tuples, and specify the Skolem functions in a 
natural manner. One familiar enumeration of the p-tuples is the follow- 


i vo 
ing: 


1.3.10 (a, , --+ , @») precedes (b;, --- , 6»). if either 


(a) they are permutations of each other but (a, --- , a,) precedes 
(b; , --+ , b,) in the lexicographic order; or 


(b) max(a,, +--+, @,) = max(b,, --- , bp), Za; = Tb; , but (a, --- 
ay), rearranged according to nondecreasing magnitude, precedes (b; 
--+ | b,), similarly rearranged, in the lexicographic order; or 


(c) max(a,,--* ,@,) = max(b,, --- ,6,), but Za; < 2b; ; or 
(d) max(a,, --- ,@)) < max(b,, +--+ , 5p). 


The Skolem functions are usually chosen by going through the infi- 
nite conjunction 1.3.9 from left to right and using each time the smallest 
unused integer for the next functional expression not yet evaluated. 
Thus, e.g., #1’, «++ , Ya,' in 1.3.7 get the constant values 1, --- , d; , and 
M, is: 


M1 o0e Ql one ld, -++ (dy + do) +++ (qq —d, + 1) --+ ql 
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Kach time a functional expression gets a value, the value is substituted 
in all later occurrences of the same expression. 

In this way we arrive at a form of the fundamental theorem of logic 
as a generalization of 1.3.3. 

It is natural to observe that the infinite conjunction 1.3.9 can be 
divided into sections (Ref. 4, p. 138): 


1.3.11 The first section is the set of those M,’s in which the p-tuples 
replacing the independent variables are made up of integers in the set 
{1, --- ,d,}, or the set {1} if d; = 0; the (n + 1)th section is the set of 
those M,’s not belonging to the nth section in which the p-tuples are 
made up of integers which occur in the union of the first n sections. 

This notion has been used by Skolem in explaining some decision 
procedures (see Section II below). 


1.4 Special Cases of the Decision Problem 


The principal known decidable classes are, with regard to satisfiability 
the following: 

I. The monadic case. The class of all formulae which contain only 
monadie predicate letters and no function symbols. 

Il. The EA satisfiability case (the AE provability case). The class of 
all formulae in the prenex form with prefixes of the form (Ly) 
(EBym)(a1) +++ (an), m, n 2 O, and no function symbols [or the form 
(yi) +++ (Ym)(Ex,) «++ (Ex,) for provability]. 

III. The conjunctive satisfiability case. Every formula in the prenex 
form with a matrix which is a conjunction of atomic formulae and their 
negations. (Equivalently, the disjunctive provability case.) 

IV. The Skolem case. Every formula in the prenex form with no fune- 
tion symbols such that it has a prefix ending with (Ly) --- (Ey,),n > 0, 
and every atomic formula occurring in the matrix contains either one 
of the variables y,, ---, yn, or all the independent variables. [For 
provability, (v1) --- (yn) at the end.] 

V. The EA2E satisfiability case (the AE.A provability case). Every 
formula containing no function symbols in the prenex form with a prefix 
(By) «++ (Bym") (a) (a2) (By?) «++ (2yn?). 

VI. The Ackermann case. For satisfiability, every formula which con- 
tains no function symbols, no equality sign, only a single dyadic predi- 
cate (G say), and has the form (x)(Hy)Gary & (2) «++ (tm)Ma, +++ tm, 
m S 4, M quantifier-free. 

In addition to these, two other cases may be mentioned: 

VII. The A,E\A, satisfiability case. Every formula with the prefix 
(x,)(EFy)(x2) and with no function symbols. 
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VIL. The Surdényi normal form case. For satisfiability, every formula 
which has no equality sign, no function symbols, only dyadic predicate 
symbols, and has the form (2;)(x2)(13)Mayrer3 & (%)(a2)(Lys)Nayroys , 
M, N quantifier-free. 

It, may be noted that in all the cases, with the single exception of ITI, 
no function symbols are permitted. Indeed, very little is known about 


the decision problem of formulae containing function symbols (compare 


Ref. 3, pp. 98-107). Unless otherwise stated, we shall always assume 
that no function symbols occur. 

In what follows, cases I and VI will not be considered. So far as the 
monadic case without equality (a subcase of I) is concerned, it is possible 
to obtain a decision procedure from one for case IT. Some of the prob- 
lems suggested by the Ackermann case are also encountered by the 
A,E,A, case, while other implications of this case seem to call for a 
closer examination of certain arithmetic predicates. 

Formulae under case VIII form a reduction class in the sense that 
there is an effective procedure by which every formula, possibly con- 
taining = and function symbols, can be reduced to one in the class with 
satisfiability preserved (Ref. 2, p. 60). It follows that there exists no 
decision procedure for this case. It is, however, desirable to find some 
“semidecision procedure” for the class which is a decision procedure for 
some subclass of it that is not specified explicitly in advance. It is thought 
that such semidecision procedures are a useful way of extending the 
range of formulae decidable by a predetermined finite set of procedures. 
A brief discussion is included in Section IV to point to the sort of thing 
which can be done along this line. It should be of interest to design semi- 
decision procedures for case VIII, as well as for other reduction classes. 

The case VII is perhaps the best known unsettled case; it has been 
mentioned in various connections (see, e.g., Ref. 11, p. 576 and Ref. 12, 
p. 420). In Section IV a procedure will be given which may be a decision 
procedure for the whole case but has only been shown to terminate for 
certain special cases. A proof of finiteness of the procedure is wanting. 
It is thought that, incomplete as the solution is, it is quite suggestive 
for further works on the decision problem. Some rather amusing com- 
binatorial problems are also related to the considerations on this case. 

An alternative decision procedure for the much-studied case V_ will 
be given in Section III in the equivalent form A, (for satisfiability). 

The Skolem case will be examined in considerable detail in Section IT, 
using ideas proposed by Skolem‘ (p. 138) and Church® (p. 264). Remarks 
relevant to machine realizations of the procedure will also be included. 

The Skolem case includes the following special cases: 
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IVa. The AjE satisfiability case. Because every atomic formula has 
to include some variable and there is only one independent variable. 

IVb. For satisfiability, every formula whose prefix ends with (Ly) 
-++ (Ey,), and in which every atomic formula contains at least one of 
the variables y; , «++ , Yn- 

IVe. For satisfiability, every formula whose prefix is 


(By!) «++ (Bym')(a1) +++ (tn)(Ey?) «++ (By?) 


and in which every atomic formula contains either all of x, , 
at least one of y;?, +++ , yx’. 

IVd. For satisfiability, every formula in the Skolem normal form, 
i.e., with prefix (21) +--+ (am)(Ey:) --+ (Eyn), such that every atomic 
formula contains at least m distinct variables. 

For the extensive literature on the decision problem, the reader is 
referred to the bibliographies in Refs. 2 and 3. The writer has not been 
able to study carefully much of the relevant literature, and is not certain 
that the procedures described in Sections II and III may not turn out 
to be inferior to existing ones. Recently, the writer noticed that ideas 
along the line of the solution of the 2A provability case given in Section 
3 of Part I' are contained in Skolem’s writings (e.g., Ref. 4, p. 135). 

Of the two remaining cases, II and III, some brief comments will 
suffice. 


oe OF 


1.5 Two Simple Cases 


The FA satisfiability case IT has agreeable decision procedures not 
dependent on the fundamental theorem of logic (see Ref. 13, p. 13). It 
is also easy to devise a decision procedure on the basis of the fundamen- 
tal theorem. Consider 


(Ey) «++ (Bym)(t1) «++ (tn) My ++ 
This is in fact equivalent to: 


1.5.2 M, & --- & Mi, k= mm or 1 when m = 0. 


b 


In fact, this is a limiting case of the fundamental theorem because no 
Skolem functions are needed, so that the m constants for the initial 
variables are all we need for fabricating a model. In other words, either 
the negation of 1.5.2 is a quantifier-free tautology, and the negation of 
1.5.1 is a theorem; or 1.5.2 has a model, and 1.5.1 has a model too. The 
presence of the equal sign is permitted, but the presence of function 
symbols in 1.5.1 would invalidate the procedure. 
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The conjunctive satisfiability case III was originally solved by Her- 
brand (Ref. 5, pp. 44-45). Suppose the matrix is: 


1.5.3 A, & -:: &€Am & ~B, & --: & ~B,, 
or, in a different notation: 
1.5.4 Aj, -+» Ana B,, ose  B. 


Assume first that neither equality nor function symbols occur. If no 
predicate letter occurs both on the left side and on the right side, then 
we can simply choose to make all predicates occurring on the left side 
true of all numbers and those on the right false for all numbers, and then 
the infinite conjunction corresponding to the given formula is true under 
the interpretation. 

Whenever there is one clause on the left and one on the right which 
contain the same predicate letter, e.g., A; is Gabe and B; is Gurw, we 
compare them and ask whether it is possible to assign the same integers 
to their arguments in some M/, and M, respectively. If the answer is 
yes, the original formula can have no model, because the infinite con- 
junction must be always false. If the answer is no for every such pair, 
then thé original formula has a model. 

To compare A; and B; , we examine the three pairs of corresponding 
variables. If both variables in some pair are distinct dependent vari- 
ables, then the two clauses A; and B; can never get the same numbers. 
When this is the case for none of the pairs, we can decide the question 
by asking whether there are positive integers s, ¢ such that a(s) = u(t), 
b(s) = v(t) and c(s) = w(t), where, for each variable a in the original 
formula, a(n) is a function giving the number which replaces a in M,,. 
It is possible to give a scheme to generate such function for each given 
formula. When there are solutions for some pair of clauses, the original 
formula is not satisfiable. 

If the formula 1.5.4 contains function symbols but not =, then the 
comparison of A; and B; has to take functions into considerations some- 
times. We may have to ask whether f(a(s)) = g(u(t)), instead of a(s) = 
u(t), has a solution. In such cases, there is a solution only when f and g 
are the same function, because otherwise we can always give different 
values to f(a(s)) and g(u(t)) to avoid the incompatibility of M, and M,. 

When the equals sign also occurs, we have to list all the equations 
among A,, --:, A,,, if there is any, and complete the list by using 
transitivity. If there are none, we need only to proceed as before, except 
that we can also reject satisfiability on the ground of, e.g., having an 


equation u = v among B,, --- , B, , and u(p) = v(p) has a solution in 
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p. In the general case, we must compare A; and B;, which have the 
same predicate letter, in a more complicated manner. One way to do 
this is to give an effective survey of all the equalities obtainable in 
M,,---,M,, for every t. And then the question of comparing Gabe 
and Guvw is reduced to the following: whether there are p, q, ¢ such 
that, with the help of the equalities obtainable from M/,, --- , M@;, we 
have a(p) = u(q), b(p) = v(q), c(p) = w(q). Since these considerations 
are only subsidiary for the main purpose of the paper, details for this 
and other steps sketched above will not be supplied. 


Il. THE SKOLEM CASE 
2.1 Outline of a General Method 


The subcase IVb, where every atomic formula contains at least one 
of the last string of dependent variables, is particularly simple. Thus, 
in every M,, each such variable always gets replaced by some new 
number so that no atomic formula in /; can have occurred in any of 
M,, +++, My. Hence, a formula of such a form is satisfiable if and 
only if ~./, is not a quantifier-free tautology. 

In the general Skolem case, we make use of the definition of sections 
given above in 1.3.11. Let (a‘, --- ,a,") be the p-tuple which replaces 
the dependent variables in / to get M/,. 

Given any member , of the nth section, the only related instances 


® ° . ° k ky - . 

in the nth section are those 7, for which (a;", «++ , a, ) is a permutation 
of (a;', --- ,a,'), and the only related instances in the (n + 1)th section 
are those WM; for which (a;’, --- ,a,’) include only numbers occurring 


in M; and at least one number not in the set {a;’,--- , a,’}. 

Hence, it is possible to get a decision procedure by determining 
whether there exists any set of possibilities which includes models for 
the instances of the first section, as well as models for allerelated in- 
stances .V, and M, for every model for 1/7; in the set. 

When the formula is in the Skolem normal form or the form of IVe, 
somewhat more is true: 


2.1.1 If WM; belongs to the (n + 1)th section, then it can have common 
atomic formulae with only at most one 7; in the nth section. 


This is so because each atomic formula in ./; either contains a new 
number not occurring in any member of the nth section, or otherwise 
contains all of {a;’, +--+ ,a@,’} with.at least one number (say a,’) which 
appeared for the first time in one specific member (say .7;) of the nth 
section. In the first case the atomic formula in .V7; does not occur in any 
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member of the nth section. In the second case, M; can contain no com- 
mon atomic formula with any member of the nth section except possibly 
M,, since a,’ does not occur in any of the other members of the nth 
section, 

Detailed considerations will be confined to the treatment of a simple 
special case. 


2.2 An Explicit Procedure for a Special Cas 


We consider a very simple special case in which the matrix contains 


no equals sign (and of course no function symbols), and a single dyadic 


predicate G: 
2.2.1 (x) (y) (Ez) Mayz. 
As an illustration, we use the negation of Example (2) of Part I:! 


2.2.2 (x){y)(E2)[(Gry & Gyr & ~Grz & ~Gzy «& ~Gzz) 
v (Grz & Gzy & Gzez & ~Gry & ~Gyx))}. 


In an alternative notation, the matrix is: 


Gary ,Gyx + Gaz,Gzy,Gzz; 


Gxz,Gzy,Gzz + Gry, Gyx. 


We construct a truth table of all the possibilities which can satisfy 
the above matrix: 


Gay Gyx Grz Ger Gyz Gzy 
t t f f 
f f t t 


The blanks may take either t or f as values. Hence, there are eight rows 
in all. 

For the prefix (x)(y)(£z), the numbers to substitute for (a,y,z) in 
M,,M2,M;3, Mg, etc., are (1,1,2), (1,2,3), (2,1,4), (2,2,5), ete. In order 
to decide whether a formula of the form 2.2.1 has a model, we ask whether 
it is possible to make W112, M123, M214, etc., simultaneously true, or, 
in other words, whether we can find for each M; one row from the above 
table according to which M, is true, such that these infinitely many rows 
are all compatible in the sense that the same atomic formula always 
gets the same truth value (t or f). 

Among the number triples we can distinguish two classes, those in 
which x and y get the same numbers, such as (1,1,2), and those in which 
they get different numbers, such as (2,1,4). The conditions under which 
a model is possible are roughly: (i) to satisfy Maab, a row in the truth 
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table has to behave in a way that x and y are interchangeable; (ii) for 
each row satisfying Mabc, there must be a related row satisfying bac; 
(iii) for the two types of row, two corresponding patterns of continua- 
tion must be possible, e.g., 


M136 
-—M123 i 


M123— —M238 


| 
| 
ia S- 995 

M225 —_—M33(10) 


These conditions can be formalized more exactly and applied, in par- 


a theorem. For this purpose, we assume a formula of the form 2.2.1 for 
which a truth table T like 2.2.4 is constructed. When, for example, Gry 
in a row R of T gets the same value as Gzz in a row S of 7, we shall use 
the brief notation R,, = 8S... 


2.2.5 A row S in the table T is a uniform row if S,, 
i. — S 


22 zy 


Clearly, for a row to satisfy 17112, it is necessary that it be uniform. 
If there is no uniform row, then there is no model for the original for- 
mula. 


2.2.6 A row S in the table 7 is an heir of a row RF in T if S is a uniform 
row and R,, = S,,. 
2.2.7 A row in T' is trivial if it has no heir. 

Since a row having no heir cannot be continued, we may cross out all 
trivial rows and be concerned only with nontrivial rows. This is not 
theoretically necessary because further requirements would cross out 
trivial rows anyhow, but it makes for efficiency. 

2.2.8 A row R in the table T is an ordinary row if there is a row S such 
that Re = Bus ® | = Say ) | <= lies 9 | = Sx ° | = Sua: « R., — 
S.,.R and S are said to be mates of each other. 


This is the condition under which R and S can satisfy (7123, W214) 
or (M214, M123) respectively. 
the two following rows are uniform rows or ordinary rows: 


Gry Gyx Grz Gex Gyz Gzy Gazz 
t t f f f f f 
B f f t t t t t 
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In fact, a and 6 are the only uniform rows, as well as the only ordinary 


rows. Each of @ and @ is only a mate of itself. 


A uniform row RF is permanent if (i) it has an heir which is per- 
manent, and (ii) there is a permanent ordinary row S such that R,. = 
S,,, R., = S,,.S is said to be a subordinate of R. 


2.2.10 An ordinary row F# is permanent if (i) it has an heir which is a 
permanent (uniform) row, (ii) it has a mate that is a permanent ordinary 
row, and (iii) there are two permanent ordinary rows P and S such that 
Res = Pay, Rez = Pyz, Rye = Sey, Ray = Syz. P and S are said to be 


a pair of subordinates of R. 


The two definitions 2.2.9 and 2.2.10 embody a simultaneous recursion. 
Condition (ii) in 2.2.9 is necessary, if, e.g., R is to satisfy 4112 and S 
is to satisfy 17123. Condition (iii) in 2.2.10 is necessary if, e.g., R 


satisfy 17123, P is to satisfy 17136 and S is to satisfy 17238. 


Is to 


2.2.11 The formula 2.2.1 has a model if and only if its truth table T 
contains a permanent uniform row. 


This assertion will be justified in 2.3. We observe first that both a 
and 8 are permanent uniform rows for the example 2.2.2. In fact, we 
have various models for the formula, which are determined, in outline, 
by the following patterns of continuation: 


B 


More exactly, choose, e.g., a as a model of 17112. As a continuation 
of this, 8 satisfies 7123 and 225; since @ is its own mate in the sense 
of 2.2.8, 6 also satisfies 1/214. Similarly, since a@ is its own mate, as a 
continuation of 8 satisfying 123, a satisfies 17136, 17317, 17238, M329, 
and 733(10). In this particular case, the model 8 of 214 can be con- 
tinued in the same way. Moreover, the model 8 of 17225 can be contin- 
ued by the row a, and, e.g., the model @ of W136 can be continued by 
the row 8, and so on. 

In the general case, a symmetry argument is needed to show that if 
a model of, e.g., 17123 can be continued, then a model of 7214 can also 
be continued. For example, if (2,8) satisfy (7123, 1214) respectively, 
and (A, B, C, D) satisfy respectively the continuation (7136, 7317, 
M238, M329) of W123, then it is easy to see that (B, A, D, C) satisfy 
the corresponding extension of 14214. This means that condition (ii) 
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of 2.2.10 can be weakened to require a mate that is an ordinary row with 
a permanent heir. 

The decision procedure implicit in the above definitions may be de- 
scribed explicitly thus: 


2.12 The decision procedure: 


1. Construct a truth table 7. 
2. Find all uniform rows. 
3. Cross out all trivial rows. 

Let U> be the set of remaining uniform rows, V» be the set of remain- 
ing ordinary rows. Each time, assume U, and V, are given and continue 
the following four steps: 

4. Eliminate every uniform row from U, which has no subordinate 
row in V,, , thus obtaining U’,,,; from U, and V,, . 

5. Eliminate from V, every ordinary row which has no mate or no 
pair of subordinate rows in V, , thus obtaining V,,,; from V,, . 

6. Eliminate every uniform row from U,,,,; which has no heir in U’,,,; , 
thus obtaining U,,2 from U4, . 

7. Eliminate every ordinary row from V,4; which has no heir in 
U.42, thus obtaining V,4,2 from V,4; and Uy42. 

8. The steps 4 through 7 are repeated until one of two things hap- 
pens: either at some stage we obtain an empty U; and an empty V;, 
then we stop and conclude that the original formula 2.2.1 has no model; 
or else, after a whole round of the steps 4 and 7, we find U’,,2 and V,42 
remain the same as U’, and V, , then we stop and conclude that the 
original formula 2.2.1 has a model. 

In practice, it is more efficient to perform, if possible, each of the steps 
4 through 8 repeatedly, before going to the next step. 

The procedure is clearly finite, since Uy and V5 are finite, and each 


round of steps 4 through 8 must reduce the size of U,, or V, if the proce- 


dure has not come to a stop yet. Moreover, the final sets U'; and V; must 
be both empty or both nonempty. 


2.3 Justification of the Procedure 


As a Skolem case, the formula 2.2.1 must not contain Grex and Gyy. 
It is, however, not obvious that we are justified in not including two 
columns Grr and Gyy in the truth tables such as 2.2.4. For a model 
constructed on the basis of such reduced tables, it is not evident that, 
for some positive integer a, Gaa might not be compelled to take on the 
value t at one place, and the value f at another. However, we can prove 
the following: ! 
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2.3.1 In every model obtained on the basis of a truth table not includ- 
ing columns for Grx and Gyy, for every number a, Gaa is never compelled 
to take on two different values. 


Take, for example, G22. If Gzz occurs in the original formula, G22 is 
compelled to take a fixed value in a model with a row R for W112. In 


the same model, if S is the row for 4225, then R,, = S,, = S,,. Hence, 
it is harmless that S,, and S,, are compelled to take the same value as 
both R,, and S,, (or S,,). In all other cases, the values for G22 can al- 
ways be given the value of R,, because there is no other place where G22 
is independently compelled to take a certain truth value. 

For the same reason, if neither an atomic formula nor any one ob- 
tainable from it by permuting the variables occurs, we may leave out 
the columns for them. For example, if Gzz does not occur, we can leave 
it out. If neither Gry nor Gyx occurs, we can leave both of them out. 

On the other hand, if, e.g., Gry and Gzy occur but Gyx does not, we 
still must include a column for Gyx. Otherwise, since we do not record 
the value of Gyx, it may happen that R satisfies 112, with R,, = t, 
and S satisfies 1/214 with S,, = f. Then no row P can satisfy 17123, 
because P,, is compelled to take both the value t and the value f, and 
this is not recorded without a column for Gyz. 

To prove 2.2.11, we remark first that there are three types of instances 
illustrated by W112, 4123, 1214. For the first kind, an 1, of the form 
Maab, the only M;, 7 > 7, which have common atomic formulae with 
M; are Mbbc, Mabd, Mbae, because these are the only ways in which 


both the independent variables x and y can be replaced by numbers 


occurring in M;, and having only one of the two arguments from M; 
yields no common atomic formula. Similarly, if M; is Mabe, a < b, there 
are only five M;,7 > 7 which have common atomic formula with M, . 
By the symmetry argument preceding 2.2.12, the mate Mbae is also 
taken care of. 

Hence, if there is any permanent uniform row, we can find a model 
for all instances .V/, , M2, ete., such that each has some common atomic 
formula with an earlier one, or, in other words, all those occurring on an 
infinite tree beginning at .¥,. This does not exhaust all the instances. 
For example, My, and M,, [i.e., .434(15) and 743(16)] are not included. 
Since, however, they contain no common atomic formulae with the in- 
stances already interpreted, we can take two permanent ordinary rows 
which are mates and get a model for another sequence of instances. In 
this way, it is seen that, if there is a permanent uniform row in the table 
T, then one can so interpret the predicate G in the domain of the positive 
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integers that the whole sequence M,, M2, ete., are simultaneously 
satisfied. 

The converse is quite obvious. If there is no permanent uniform row, 
then no interpretation of 14112 can be continued indefinitely, and there 
is an 7. such that M, & --- & M; is true under no interpretation. 


2.4 (Questions of Efficiency 
When doing an example by hand, there are shortcuts we find natural 
to use. These may be viewed as more refined methods which can be 
mechanized by additional efforts. We give some informal illustration of 
the type of quick method we tend to use. 
Consider the negation of Example (3) given in Part I:! 
2.4.1 (x)(y)(E2z){[Gary & (~Gyz v ~ Gzz)| 
v [(Gry & Hry) & (~Hxz v ~ H2zz)}}. 
In the alternative notation, the matrix of the above formula is: 
2.4.2 Gary + Gyz; Gry + Gzz; Gry,Hay + Harz; Gry,Hay + Hz2z. 
The truth table for this is: 
24.3 Gry Hxry Gyr Hyx Harz Her Gyz Gey Geez Hz 
a t f 


t 
t t 
t t f 


Although the formula contains two predicates instead of just one, it 
is easy to see that the procedure described above can be extended to 
cover the case in a very straight-forward manner. 

Since there are many blanks in the table, it is essential for efficiency 
that we do not expand the table by filling in the blanks (there would 
be 2 rows), until we are compelled to do so. In other words, we try 
to carry out the decision procedure by treating each row containing 
blanks as a single row and make expansion only when we are not able 
to eliminate them as single rows. 

We observe that for every row, in particular, every uniform row, Gry 
gets the value t. It follows that row 8, or more exactly, all the 2’ rows 
obtainable from 8 are trivial by 2.2.7, since an heir of 6 must have Gry 
take the value of Gzz in 8, which is f. Hence we may delete row 8 alto- 
gether. 

In order that row a, or any specification R of a, be permanent (uni- 
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form or ordinary), it is necessary, by 2.2.9 and 2.2.10, that there is a 
subordinate row S, such that Gry gets the same value in S as Gyz in R, 
or Rey = Ser,. But this is impossible because R,,. is f in every row 
obtainable from a, but Sg, is in every row. Hence, we can delete row a 
altogether, and be concerned only with the rows y and 6. 

Since Hxry gets t in all the remaining rows and Hzz gets the value f in 
5, every row obtainable from 6 has no heir, and the whole row 6 can be 
deleted. 

However, no permanent ordinary row can be obtained from y alone 
because, by 2.2.10, for any such row R there must be a subordinate row 
P such that Ry,, = Pys,, but in row y, Hxz is always f and Hry is 
always t. Hence, there can also be no permanent uniform row, and, by 
2.2.11, the formula 2.4.1 has no model. Therefore, Example (3) in Part 
I,' the negation of 2.4.1, is a theorem. 

Another method of deciding 2.4.1 is the following. We begin with 
M,, which is a disjunction of conjunctions, and choose M;, M; , ete., 
which contain common atomic formula with 1, in the hope that 
M, & M; & M; & --+ as multiplied out into a disjunction of conjunc- 
tions will include in each conjunction some atomic formula and its 
negation. The process may have to be continued. 

As we observed before, only M,_, WM; , WM, can have common atomic 
formulae with M,. Of these three, on account of the special structure 
of 2.4.1, 3 has no common part with W, . Hence, we need to consider, 
to begin with, only WM, , M., M,: 

(i) (il) (ili) (iv) 
M112 G14 G12, 11-4 G22; G11, H11-4 H12; ~—@11, W114 H22 
M123 G12 + G23; G12 + G33; G12, H12 + H13; G12, H12 + H33 
M225 G22 ++ G25; G22 +> (55; G22, H22 +> H25, 722, H22 ++ H55 

By the row for M123, (i) of M112 can be deleted because (i) contains 
~@12 (i.e., after +), while each clause in the row for M123 contains 
G12. It can be seen then that every row in column (i) can be deleted 
in the same way. Similarly, (ii) of the row for M112 can be deleted 
because it contains ~G22, while each clause in the row for M225 con- 


tains G22; therefore, the whole column (ii) can be deleted eventually, 


and we need only consider the columns (iii) and (iv). But then (iii) of 
the row for M112 can also be deleted because it contains ~H12, and 
all the remaining columns of the row for W123 contain H12. Finally, 
we have only column (iv) left. Now, however ~H22 occurs in the row 
for 17112 and H22 occurs in the row for 4225. Hence, the conjunction 
of the three rows of column (iv) is a contradiction, and 2.4.1 has no model. 
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2.5. The Inclusion of Equality 


The decision procedure in 2.2 can be extended to deal with cases where 
the equal sign occurs in the given formula: 


2.5.1 (x) (y) (Bz) Maye, with = occurring. 


Additional considerations are needed to take care of the special 
properties of =. First we bring Mryz into a disjunction of conjunctions 
of atomic formulae and their negations, in the usual manner. Then we 
modify the resulting matrix to take care of the properties of =. (a) 
Kach conjunction that contains an inequality of the form v # »v, v being 
x or y or 2, is deleted. (b) In each conjunction, a clause of the form 
v = v is deleted. (c) Within each conjunction, if vu = v is a clause with 
distinct variables u and v, we add also, as new clauses (if not occurring 
already), » = u and the result of replacing any number of occurrences 
of u by v (or v by wu) in each clause of the conjunction; this is repeated 
for every equality until no new clause is generated. (d) Repeat the steps 
(a) and (b) on the result obtained by step (c); in addition, any conjune- 
tion which contains both an atomic formula and its negation is deleted. 

We now construct the truth table on the basis of the new matrix (in 
a disjunctive normal form). Uniform rows, ordinary rows and _per- 
manence can be defined in a similar manner as before, except that a 
uniform row has to satisfy the additional condition that « = y and y = x 
both get the truth value ¢ (not only that they just get a same value). 
In this way, we can obtain a decision procedure for all formulae of the 
form 2.5.1. 

It is believed that the same type of consideration can be used to 
extend all the cases considered in this paper to include also the equal 
sign. In the next two sections, equality will be left out and attention 
will be confined to formulae not containing the equals sign (nor, of 
course, function symbols). 


lll. THE Aof SATISFIABILITY CASE 
We give an alternative treatment of this case which, it is conjectured, 
is in general more efficient than the method of Schiitte' as reformulated 
a 8 ry . . ° ° 
by Klaua.” The method will be explained with the special case when only 
one dependent variable and only one dyadic predicate G occur: 


3.1 (x) (y) (Bz) Mryz. 


The main difference between this case and the case solved in 2.2 


above is that Grx and Gyy are permitted to occur in Mayz. As a result, 
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for example, W123 may contain common atomic formula with any 
Mabc in which a or b is one of 1, 2, 3. 
As an example, we choose arbitrarily the following: 


3.2 (x)(y)(E2)[~Gre & (Gry D ~Gyx) & Grz & (Gzy D Gry)). 
The matrix may be rewritten as: 


3.3 Garz + Grx,Gey,Gzy; Gez,Gry + Grx,Gyx; 


Guz + Grxr,Gyz,Gzy. 
The truth table is: 


3.4 Gixx Gry Gyx Gyy Gaz Gyz Ger Gey Gz 
t f 
t f 
t 


The problem is, as before, to decide whether there is a model that 
satisfies M, , M, , ete., simultaneously. The conditions are rather similar 
to those in 2.2 except that for any two rows R and S which, say, satisfy 
Mabe and Mdef in a model, there must be two rows which satisfy Mefg 
and Mfch in the model. There is also a related requirement for a row 
satisfying 1, , because the number | is never used to replace a depend- 
ent variable. The various conditions may be stated: 


3.5 A row R is uniform if R22 = Ri, = Ryz = Ry, Ra = Ry, Rar = Ry. 
A row S is an heir of a row R if S is uniform and R., = S, 
Two rows R and S form a parallel pair if R,, = S,,, R2, 


Ses ’ Ten aa S z 9 R,- = Sas ’ = — 


yz / 


= Sia ; R 


7h] 


Two rows of a parallel pair are said to be mates of each other. 
If R and S are to satisfy Mabe and Mbae, it is necessary that they form 


a parallel pair. In general, for a row satisfying Mabc, there must also 
be two parallel pairs of related rows satisfying Macd, Mcae, Mbef, 
Mcbg. When a = b, the two parallel pairs become one. This, plus the 
requirement that every row in a model must have an heir may be sum- 


marized in the following condition. 
3.8 A row R is normal if the following conditions are all satisfied: 
3.8.1 It has a normal row as a mate; 


3.8.2 It has an heir which is a normal row: 
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3.8.3 There are two normal rows P and S such that R.. = P.2, RR. = 
Pn ’ R., a Pros ’ Ru _ Pw ’ and Ry _ Bing , R,: - Bus , Bus - Sess , 
R., = S,,. Such rows P and S are said to be subordinates of R. 


A uniform row is its own mate, although a self-mated row is not al- 
ways a uniform row. For a uniform row, 3.8.1 is a redundant condition, 
and P and S coincide in 3.8.3. The definition 3.8 of normality is clearly 
recursive. 

In the table 3.4, we observe that, because Grx always takes the value 
f, Gzz can only take the value f in order that the row has an heir. More- 
over, since Grx always gets the value f and Gxz always gets the value t, 
in order that a row has a mate, Gyy must always take the value f and 
Gyz always t. Hence, we need consider only the following eight rows 
which result from filling the remaining gaps: 


( 


~ 
“a 
~ 
~ 


3.9 


Gyy Grz Gyz rex | Gzy = Gzz 
f t 


re 
f 
f 
t 
f 
t 
f 
t 
t 


t f i 
f t t f f 
f t t f f 
f t t f f 
f t t t f 
f t t f f 
i t t f { 
f t t t f 


Row e has no mate, beacuse of the columns 5 to 8. Rows ¢ and d have 
no mate, because b, the only row satisfying the condition on Gzx and 
Gzy, does not satisfy the condition on Gry and Gyx. Neither row a nor 
row b has subordinates as required by 3.8.3. Hence, we have only the 
remaining rows a, 6; , 82 to consider. 

a is the only uniform row, (8; , 82) form a parallel pair; and 8» is both 
P and S in 3.8.3 for all the three rows a, 8; , Bo. Hence, we have, for 
example: 


—( M225, a) 


(M33(10), a) 


| (M135, Be) 
(M112, a) é (M123, Be) 
| 
} 


} 


M316, B, 
M237, Bo 


M328, 8;) 
(M214, 8:) 
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In particular, (.7214,6,) can be continued in the same way as (1/123, 
82). Indeed, continuation in every branch can be made similarly. In 
other words, a, 8;., 82 are all normal by 3.8. This, however, does not yet 
secure a model for the formula 3.2. There are, for example, those in- 
stances in which (1,5), (5,1), (3,4), (4,3), ete., replace (x,y) of Maxyz; they 
also have common atomic formulae with the instances shown in the 
above graph. 


3.10 A formula 3.1 has a model if and only if (a) it has a nonempty table 
of normal rows, (b) this table has a nonempty subtable 7” such that: 


3.10.1 For every pair (R,S) in 7’, there is a parallel pair (P,Q) in T’ 
such that P., = R,, , Q22 = Sz. 


3.10.2 There is a uniform row R in T’ such that for every row S in T’, 
there is a parallel pair (P,Q) in T’, for which P,, = R,,, Qrz: = Su. 


These are the additional requirements mentioned after 3.4. In the 
example under consideration, the table consisting of all the three normal 
rows a, 8; , 82 satisfies the requirements on 7’. Hence, 3.2 does have 
models. One model for the predicate G is the relation < among positive 
integers. That is, however, not the only model, because the model of G 
does not have to be transitive. For example, G15 and G51 can be (t,f) or 
(f,t) or (f,f). 

It can be verified that the conditions in 3.10 are indeed necessary and 
sufficient. 


IV. THE A,/\A, SATISFIABILITY CASE 
4.1 A Generalized Game of Dominoes 


The study of the decision problem of the present case has suggested a 
related abstract mathematical problem which can easily be stated in 
everyday language. The problem appears to be of interest even to those 
who are not concerned with questions in mathematical logic. 

Assume we are given a finite set of square plates of the same size with 
edges colored, each in a different manner. Suppose further there are 
infinitely many copies of each plate (plate type). We are not permitted 
to rotate or reflect a plate. The question is to find an effective procedure 
by which we can decide, for each given finite set of plates, whether we 
can cover up the whole plane (or, equivalently, an infinite quadrant 
thereof) with copies of the plates subject to the restriction that adjoining 
edges must have the same color. 

For example, suppose a set consists of the thuee plates: 
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Then we can easily find an infinite solution by the following argument. 
The following configuration satisfies the constraint on the edges: 


a 2 ¢ 
CA S&S 
BC A 


Now the colors on the periphery of the above block are seen to be the 
following: 


3.5 4 


In other words, the bottom edge repeats the top edge, and the right 
edge repeats the left edge. Hence, if we repeat the 3 X 3 block in every 
direction, we obtain a solution of the given set of three plates. In gen- 
eral, we define a “cyclic rectangle.” 


4.1.1 Given any finite set of plates, a cyclic rectangle of the plates is a 
rectangle consisting of copies of some or all plates of the set such that: 
(a) adjoining edges always have the same color; (b) the bottom edge of 
the rectangle repeats the top edge; (c) the right edge repeats the left 
edge. 

Clearly, a sufficient condition for a set of plates to have a solution is 
that there exists a cyclic rectangle of the plates. 

What appears to be a reasonable conjecture, which has resisted proof 
or disproof so far, is: 


4.1.2 The fundamental conjecture: A finite set of plates is solvable (has at 
least one solution) if and only if there exists a cyclic rectangle of the 
plates; or, in other words, a finite set of plates is solvable if and only if 
it has at least one periodic solution. 

It. is easy to prove the following: 


4.1.3 If 4.1.2 is true, we can decide effectively whether any given finite 
set of plates is solvable. 


Thus, we proceed to build all possible rectangles from copies of the 
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plates of different sizes, using smaller ones first. If 4.1.2 is true, the 
process will always terminate in one of two ways: either at some stage 
we arrive at a cyclic rectangle and, therefore, the original set is solvable; 
or else we arrive at a size such that there is no rectangle of that size in 
which adjoining edges always have the same colors. The latter alterna- 
tive is in fact a necessary and sufficient condition under which the 
original set is not solvable. However, if 4.1.2 is not true, it would be 
possible that a set has a solution, but we can never see this fact by the 
latter criterion at any finite stage: there would always be the possibility 
that for the next size there exist no rectangles with same-colored ad- 
joining edges. 

There is a naturally uneasy feeling about the effectiveness of such a 
procedure. The argument is essentially the familiar one that if a set and 
its complement are both recursively enumerable, then the set is recursive. 
It shows that the procedure always terminates (provided 4.1.2 is true) 
but gives no indication in advance as to how long it might take in each 
case. 

If 4.1.2 is proved, it seems likely that it would be proved in a stronger 
form by exhibiting some simple recursive function f with the following 
property. For any set of plates with m distinct colors and n distinct 
plates, if the set is solvable, there is a cyclic square of the size k X k, 
where k = f(m,n). If that happens, or even if we have not exhibited such 
a function f but 4.1.2 can be proved by fairly elementary arguments, we 
would have some estimate in advance of how long the procedure takes in 
each case. 

As it is, we can make the testing procedure quite systematic even 
though we do not know whether 4.1.2 is true. The procedure would be a 
decision procedure and presumably quite an efficient one, if 4.1.2 is true. 
If 4.1.2 should turn out to be false, then the procedure would only be a 
semidecision procedure. In fact, it is possible to show that the procedure 
does work in several classes of cases, e.g., when a set has unique solution 
apart from translations, or whenever either horizontally or vertically no 
color can be followed by different colors. But we shall not delay over 
such partial results. 

If 4.1.2 should be false, then there would be two possibilities: either 


the set of all solvable finite sets of plates is not recursive, or it is recursive 


but requires a more complex decision procedure. 

The problem can clearly be generalized to higher dimensions: for 
example, to cubes with colored surfaces instead of squares with colored 
edges. 

We return now to the A,/A, satisfiability case. 
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4.2 Preliminary Definitions and an Example 
The general form of the case is: 
4.2.1 (x) (Ly) (z)Mayz, 


where M is a quantifier-free matrix containing neither function symbols 
nor the equality sign. From the fundamental theorem, it follows that 
4.2.1 is satisfiable (solvable) if and only if each finite subset of the infinite 
set of matrices Mii’) (t,7 = 1, 2, ---) is solvable (not contradictory). 
Since the second number is always the successor of the first, we shall 
write Mij for Mit’). 

We illustrate the general case by considering the special case where 
Meyz contains only a single dyadic predicate G. The negation of Example 
(4) given in the introduction of Part I! will be the concrete example: 


4.2.2 (x)(EBy)(z)[~Gre & Gry & (Gyz D Grz)). 
In the alternative notation, the matrix is 

4.2.3 Gay ,Grz + Gre; Gry + Gyz,Grx. 
The truth table is: 


Gax Gry Gyr Gyy Grz Ger Gyz 
f t t t 
f t t f 
f t f f 


Since there are five blank columns, there are altogether 3 X 2° or 96 
rows. The problem now is to decide whether we can choose one row for 
each matrix Mij(i,j = 1, 2, ---) such that, taken together, all the 
matrices come out true. This really involves both the problem of finding 
the pieces and the problem of putting them together. Thus, if 7 is dis- 
tinct from 7 and 7’, any row can satisfy Mz alone, if we substitute 7, 
i’, j for x, y, z in the truth table; but a row can satisfy M7j when 7 is 7 
or 7’ only in case certain related columns get the same truth values. This 
is the problem of finding the pieces. When there are such pieces, there is 
the harder problem of putting them together. For example, if there are 
rows satisfying M11 and M12 separately, there may yet be no pair of 
rows which satisfy M11 and M12 simultaneously because the common 
atomic formulae in both matrices must get identical values. 

Since the putting-together part is quite complex, it seems natural to 
combine small pieces into blocks first. For this purpose, we consider row 


pairs and row quadruples (i.e., pairs of pairs). 
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p4.1 Two rows P,Q in the truth table T form a basic row pair (P,Q) 
if, for some 7, they can simultaneously satisfy Mii’ and Mii respec- 
tively. More explicitly, the conditions are: 

. &= F @ Py @ Pei Tig @ Pai hee OT Feet 

lM. Qez = Qee = Qee = Qee, Vey = Vy, Vz = Ve 

tee? Oo ra @ Oe, Fee =e te Oe ee 

In.the table 4.2.4, it is easy to verify that there are only two basic 


row pairs (a,8) and (y,6): 


4.2.5 Grx Gry Gyr Gyy Grz Ger Gyz Gey 


f f f f 
f f t f 
f t t { 
f f t f 


f t f f t 
f t f f 
f t f t t 
ae t f t f 

Obviously basic row pairs are necessary for building a model of 4.2.1. 
In fact, given any formula 4.2.1, if its truth table 7' contains no basic 
row pairs, then it has no model and, indeed, the conjunction of 4/11 and 
M12 is a contradiction. 

We shall consider pairs of row pairs, called row quadruples, which are 


useful in chaining row pairs together. 


p4.2. Given any two row quadruples (A,B; C,D) and (P,Q; R,S), if 
C = P, D = Q, then the former is a predecessor of the latter and the 


latter is a successor of the former. 


p4.3. Four rows P, Q, R, S form a basic row quadruple (P,Q; R,S) if, 
for some 7, they satisfy simultaneously Mii’, Mii, M7i'i”, Mi'v’, respec- 
tively, or, more explicitly, if: 

i. (P,Q) and (R,S) are basic row pairs; 

S. Pug ae + 

iii. (P,Q; R,S) has a successor which is a basic row quadruple. 

In the table 4.2.4, there is only one basic row quadruple, viz., (@,8; 
a,8). The quadruple (a,8; 7,6) satisfies i and ii, but not iii. It is easy to 
see that, given any formula 4.2.1, if its truth table T contains no basic 
row quadruples, then it has no solution and, indeed, the conjunction of 
M12, M11, M23, M22, 34, M33 is a contradiction. 

Clearly, if a row R satisfies /7j’ in a model, then there must be one 
row S which satisfies ji, one basic row quadruple (A,B; C,D) which 
satisfies Mii’, Mii, Mi'i”, Mi’t’, and one basic quadruple which satisfies 
Mjj’, Mjj, Mj'j”, Mj’j’. In particular, when 7 is 7, we get the basic row 
pairs which occur in some basic quadruple. 
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p4.4 Two rows R,S form an ordinary row pair (F,S) if 
i. Reo = Sus, Ree = Su, Res = Sus, Rea = Su; 

ii. There is a basic quadruple (A,B; C,D) such that A,, 
Ag = Ry, Ayz = Ryz, Aw = Ry; 

ili. There is a basic quadruple (P,Q; K,L) such that P,, 
Pry = Say, Pyz = Syz, Pw = Sy. 

In the table 4.2.4, since the only basic quadruple is (a,8; a,8), it is 
relatively simple to find all the rows which do occur in ordinary row 
pairs. Since every row which is to satisfy some 7) in any solution must 
occur as one row in some ordinary row pair, we tabulate all such rows 
together and, from now on, confine our attention to them. It happens in 
this example that all these rows have in common five columns: 


Gux Gry Gyx Gyy Gazz 
f t f f f 


Therefore, we only have to list the remaining columns: 


4.2.6 x Grzx Gyz Gizy ordinary 
patrs 
(a,8) 
(B,a) 
(6; , 51) 
(52 , 52) 
(3 , 63) 
(65 , 45) 
(65 , 64) 


(66 , 67) 


In fact, if only the four columns have to be considered, there are 12 
rows in the original table 4.2.4, and the two rows (R,S) in each ordinary 
row pair satisfy the condition: R,, = S,,, R:. = S,.. Hence, it is easy 
to get the above table. Briefly, the relevant information for the example 
is the nine ordinary pairs given above and the basic quadruple (a@,8; @,@). 

Thus far we have been concerned only with rather elementary proper- 
ties of the rows in the truth table. The more involved part is to design a 
scheme of extending recursively the construction of models. In order to 
explain how this is done, we introduce a chart. 





28 THE BELL SYSTEM TECHNICAL JOURNAL, JANUARY 1961 


4.2.7 Chart for (x)(Ey)(z)Mayez: 
Basic Pairs Cyclic Pairs Common Row Pairs 
(Mii’, Mit) = (Mii”, M7’t) (Mi(t” + k), M(i’ + k)i) 
(12,11) —— (13, 21) a >» (15,41) — (16, 51) 


a a om 


¥ . 4 . / 
(28, 22) <> (24, 32) ‘ 25, 42) —— (26, 52) mid 


fr | A 


f 
7 


(34, 33) > (35, 43) > (36, 5 > 
7 
(45, 44) ——> (46, 54) 


SA 


(56, 55) 


In the chart, the ordinary row pairs satisfying (.W/7j’, Mji) are divided 
into three classes: basic when i = j, cyclic when 7’ = 7, common other- 
wise. The general plan of the procedure is as follows. The existence of 
basic row quadruples assures that we can find a model for all the matrices 
M12, M11, 23, M22, etc. in the first column. Similarly, we can define 
cyclic quadruples to give an effective condition for the existence of a 
model for all matrices appearing in the second column of the chart, and 
so on. But in order that these models can be combined to give a model 
for all the matrices and therewith for a given formula 4.2.1, each column 
must be related to the column on its left in a suitable manner. This 
situation with two infinite dimensions seems to be the chief cause of the 
complexity of the A,/\A, case. 

In the chart of 4.2.7, each row pair (R,S) that is not basic is subor- 
dinate to a quadruple (A,B; C,D) made up of the two row pairs (A,B), 
(C,D) on its left with arrows leading to it. The quadruple is said to be 
superior to the pair (R,S). 


p4.5 An ordinary row pair (R,S) is a subordinate of a quadruple 
(A,B; C,D) if 

Bee ™ Ans Ba ™ Ans Bee * Bony Ome 
a ee ee 

Bi Meee: Den: Beg Deg y Ben Digs Baa Bi Be ™ Mae's 
A quadruple (R,S; P,Q) is subordinate to a row sextuple (A,B; C,D; 
K,L) if (R,S) is subordinate to (A,B; C,D), and (P,Q) to (C,D; K,L). 


p4.6 Two rows R,S form a cyclic row pair (R,S) if 
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i. (R,S) is an ordinary row pair; 

i Ry = 6; Be = 8a, 8 = Bic, Re @ Bes Ra = Sys - 

Obviously, given 4.1, if its table contains no two rows forming a cyclic 
pair, then the conjunction, briefly Cs , of 12, M11, 423, M22, M13, 
M21 is a contradiction. 

In the table 4.2.6, there are, among the nine ordinary row pairs, only 
one that is cyclic, (6,, 55). Since there are only one basic quadruple, 
each has only one superior. This is of course not always the case, it is 
only due to special features of the example 4.2.2. 

In order to find out whether there is any succession of cyclic pairs 
which will satisfy all rows of the column for cyclic pairs in the chart, we 
study cyclic quadruples. 


p4.7 Four rows P,Q,R,S form a cyclic quadruple (/?,Q; R,S) if 
i. (P,Q) and (R,S) are cyclic row pairs; 

li. Qee = Rez, Qey = Rey, Qs = Ryz, Gy = Rw; 

ill. There is a basic sextuple (A,B; C,D; K,L); which is respectively 
superior to (P,Q; R,S); 

iv. (P,Q; R,S) has a successor which is also a cyclic quadruple. 

Obviously, given a formula 4.2.1, if its table contains no rows that 
form a cyclic quadruple, then the conjunction of C, , 434, 133, 724, 
M32 is a contradiction. 

The existence of a cyclic quadruple certainly assures that we can 
satisfy all the rows of the second column of the chart simultaneously. It 
assures a bit more: the two pairs (P,Q), (R,S) of a cyclic quadruple are 
always compatible with any three pairs (A,B), (C,D), (K,L) which form 
two basic quadruples, respectively superior to them. This is, however, 
insufficient to secure that all the rows in the first two columns of the 
chart can be simultaneously satisfied, because it is possible that no 
cyclic quadruple beginning with (R,S) is subordinate to any quadruple 
beginning with (K,L). In other words, the blocks might not fit together. 


there is only one cyclic pair (4,4, 5s), there can be at most one cyclic 
quadruple, viz, (64, 55 ; 54, 55). It can be verified by D4.7 that this is 
indeed a cyclic row quadruple. Since there is only one basic quadruple 
(a,8; «,8), we see immediately that by using (a@,8) for (Mii’, Mii) 
(i = 1, 2, ---) and (6,, 6;) for (Mii”,M7i’1)(¢ = 1, 2, ---), all these 
matrices (of the first two columns of the chart) are simultaneously satis- 
fied. Moreover, this is the only possible model for the two initial infinite 
columns of matrices. 

We shall first define common row quadruples, settle 4.2.2, and then 
come back to the more general question. 
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p4.8 Two ordinary row pairs (R,S), (P,Q) form a common quadruple 
(RS; P,Q) of order k [i.e., in the (2 + k)th column of the chart] if 

i. When k = 1, there is a cyclic row sextuple which is superior to 
(RS; P,Q); or when k = n + 1, for some positive integer n, there is a 
common row sextuple of order n which is superior to (R,S; P,Q). 

ii. (R,S; P,Q) has a successor which is a common quadruple of order k. 

By this definition, we can successively find the common row quad- 
ruples of orders 1, 2, etc. In the actual procedure, we examine each time 
to determine whether we have already enough information to decide the 
original formula. Only when this is not the case do we find the common 
quadruples of the next order. 

In the case of 4.2.2, since (6, , 55 ; 64 , 5s) is the only cyclic quadruple, 
it is easy to verify, by 4.2.6 and D4.5 that (6, 5; ; 6,, 55) is the only 
common quadruple of order 1. Thus, by D4.5, if (R,S) is subordinate to 
the cyclic quadruple (5, , 55 ; 64, 55), Ry. = (6s)22 = t, Ry = (622 = f, 
and S,, = (64).2 = f, Sic = (64), = t. By 4.2.6, (R,S) must be (64, 55). 

From this, it follows that, for every n, there is exactly one common 
quadruple of order n, viz. (64, 55 ; 64, 45). This is an immediate conse- 
quence of D4.8 and the above transition from the cyclic column to the 
first common column in the chart. Hence, we have obtained a model for 


relation < among positive integers. 

This completes the solution of the example 4.2.2, which, however, is 
not a sufficient illustration of the general case. We have to discuss a 
procedure by considering more complex situations. 


4.3 The Procedure 


One possible procedure is to add one infinite column at a time. Thus, 
it is possible to represent all possible solutions of each column by a 


graph, and to represent the solutions satisfying all the initial n’ columns 


by a finite set of graphs if it is possible so to represent all solutions satis- 
fying the initial n columns. Since the common columns enjoy a measure 
of uniformity, simultaneous solutions for all the columns would be as- 
sured if suitable repetitions occur. An exact explanation of such a pro- 
cedure would be quite lengthy. In any case, a successful choice of pat- 
terns of repetition has not been found to assure that for every solvable 
table, such repetition always occurs. 

Instead of elaborating the above procedure, we transform the problem 
to something similar to the abstract question of 4.1. Thus, given any 
formula of the form 4.2.1, we can, as in 4.2, construct its truth table and 
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find all the common row pairs in the table. Among the common row 
pairs, some are also cyclic row pairs and some are also basic row pairs. 

If now we take the common row pairs a, b, c, d, etc., as elementary 
units which are to fill up the infinite quadrant as shown in the chart 
given under 4.2.7, then the following scheme appears to be feasible. Sup- 
pose the points in the infinite quadrant are to be filled by aj; , 7,7 = 
1, 6, --- , then we may consider instead all the 2 X 2 matrices: 

ay Ais’ 

for all 7,7 = 1,2,--- 

Aji = Ajrjr 
In other words, given the common row pairs, we can form all possible 
2 X 2 matrices of them which satisfy the relations of subordination. 
These 2 X 2 matrices are then the basic pieces from which we are to 
obtain an infinite solution subject to the conditions: (a) consecutive 
rows or columns from two matrices are the same; (b) only basic and 
cyclic row pairs are permitted in the first two columns. 

It can be verified that the problem of finding a model for the original 
formula is equivalent to that of finding a way to fill up the infinite 
quadrant by such derived 2 X 2 blocks of row pairs. 

The abstract problem is: given any finite set of 2 * 2 matrices of the 
form 

‘ 


, 


b d 


to decide whether it is possible to fill up the infinite quadrant with copies 


of these pieces. This is not quite the same as the problem of colored 
plates described in 4.1, because here what is done amounts to coloring 
the corners, or imposing connections between neighboring sides within a 
same square. 

Any set of such 2 X 2 matrices can also be construed as a set of colored 
plates. Conversely, given any set of colored plates, we can also find in a 
systematic manner a corresponding set of such matrices such that the 
solvability problems for them are equivalent. For example, we may 
replace a colored plate by a block of nine 2 X 2 matrices so that the 
restriction on neighboring sides no longer operates. 

It is possible to use a procedure similar to the one described roughly 
in 4.1. Some change is needed to take care of the additional conditions 
on the first two columns. Thus, a sufficient condition is to get a eyelic 
rectangle m X non which we can attach a frill of two columns on the 
left to obtain a rectangle m X (2 + n) such that: (a) the tops of the 
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first two columns are the same as the bottoms; (b) the additional re- 
quirements of being basic or cyclic are satisfied by the frills. 


4.4 Further Problems 


The discussions so far seem to have barely scratched the surface of a 
group of rather difficult problems, among which the basic one is probably 
that of measuring the complexity of formulae in the predicate calculus. 

One may measure the complexity of a formula in many different ways. 
The “simplest” model of a formula may be taken as a semantic measure. 
The quantifier prefix or graph of a formula may be taken as a syntactic 
measure. In addition, for formulae with a same prefix, we may also 
classify the possible matrices by the truth tables. Our knowledge on 
using these criteria to give detailed classifications seems very limited. 
One example of the ignorance is the following open problem (Ref. 2, p. 
177): whether there is any class of formulae which is neither decidable, 
nor a reduction class. It appears reasonable to conjecture that there 
must be such classes, although the first examples which one will get are 
likely to be artificial ones. 

Some of the reduction classes are, formally speaking, surprisingly 
simple. For example, from the Surdnyi normal form given above as case 
VIII, it follows that, for satisfiability, one reduction class is: 


4.4.1 Formulae with prefix (x)(y)(£z)(w)Maxyzw, where M contains 
neither function symbols, nor =, nor predicate letters which are not 
dyadic. 

Since each matrix M is effectively determined by a truth table on the 
atomic formulae in M, the class may be viewed as a union of a simple 


sequence of finite classes C; , C2 , ete., where C, is the subclass of formulae 


ach containing exactly n predicates (or, equivalently, the first n pred- 
icates in some enumeration). There is a sense in which the decision 
problem for each finite set of formulae is solvable, and yet usually we as a 
matter of fact only solve the problem as a corollary to a solution for some 
infinite class. 

To obtain a semidecision procedure for the class VIII or 4.4.1, we 
need more complicated arrangements of triples or quadruples of positive 
integers than the case A,/,A,. Take, for example, the class in case 
VIII. We have to consider not only the triples (a,b,c) with b = a’, but 
all the triples for the first half of the formula, and among them those 
for the A.F; case are used simultaneously for the second half of the 
formula. 

An example is: 
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4.4.2 (x)(y)(z)(~Gry v ~Gyz v Gaz) & (x)(y)(Eu)(~Gar & Gyu). 


If we use the Skolem function g of the A,/; case, we can rewrite the 
above as 


4.4.3 (~Gry v ~Gyz v Gaz) & (~Grx & Gygry). 


In general, we are concerned with deciding the satisfiability of formu- 
lae of the form 


4.4.4 Meayz & Narygxry. 


As (x,y,z) runs through all triples of positive integers, we get an infinite 
sequence from 4.4.4, and a semidecision procedure is to decide, for 
certain cases, whether such an infinite sequence can be simultaneously 
satisfied. 

For example, we may throw together all permutations of a given 
triple, and confine ourselves to the triples (a,b,c) with a S b Cc, as- 
signing each of them a lattice point: 


f(z,y,z) = (a -—1l,z—2,2-y), 
f(a,y,2) (@+tileatyrt+yt2). 
The correlation uses all lattice points (x,y,z) of nonnegative integers. 
For instance, (1,3,5) gets the point (0,2,2). 

We might try to create different types of cubes each with eight ver- 
tices from (i,j,k) to (7’,7’,k’) and piece them together. But it is not easy 
to sce how to find a procedure analogous to that described in 4.1 which 
would at the same time take into consideration the second half of the 
formula. 


V. A PROOF PROCEDURE FOR THE PREDICATE CALCULUS 


5.1 The Quantifier-Free Logic F 


Given the definition of formulae in 1.2, we can define sequents, ante- 
cedents, consequents, as in Ref. 13, p. 5. The sequents in F are those 
containing no quantifiers and the rules for F are exactly the same as 
those for P, (Ref. 13, p. 8), except for containing not only variables but 
also functional expressions as terms. 


Example 1.1 ¥ z’,x =2x2+17142+1 


By the rules P2a and P2b (Ref. 13, p. 5), this is a theorem if the 
following is: 


l=2r+127?=272+1-1= 2’. 
This is a theorem by P7 and P8 (Ref. 13, p. 8). 
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Example 2.x + y’ = («+ y)'",yAatyy =v’ Dy=r,v= 
etyoy Arty 


By P2a, P2b, and P5b, this is a theorem if the following two sequents 
are: 


Laty=(c#t+y)’,y=v,v=rtyy =atyoy=rty; 
lic ty =(@#+y)vH=retyy =aet+ yoy =v,y=rt+y. 


i. isa theorem by P7 and P8 since we can replace y and x + y by v. 

ii. is also a theorem because we can replace v’ by (« + y)’ and then 
y’ by x + y’ in the first clause of the consequent and the result is a 
theorem by P1. 

These rules in fact yield a decision procedure for all quantifier-free 
sequents. In order to see this, we use a more efficient method to speed 
up applications of P7 and P8. 

Given an atomic sequent which contains equality but is not yet a 
theorem by P1 or P7. List every pair (a,b) if a = b occurs in the ante- 
cedent. Extend repeatedly the set of pairs by symmetry and transitivity. 
Join each pair by the equals sign and add all of them to the antecedent. 
Now compare each clause in the antecedent with each clause in the 
consequent to see whether there is a pair of clauses which can be ob- 
tained from each other by substituting equals for equals; moreover, 
examine each equality in the consequent to see whether it can turn into 
a = a by substituting equals for equals. If either case occurs, the 
sequent is a theorem. If neither is the case, then we can find an interpre- 
tation of the functions and predicates so that the antecedents are all 
true but the consequents are all false. 


5.2 The Rules for Quanitifiers. 


In the present formulation of the predicate calculus, one emphasis is 


on separating out reversible rules of proof which serve to supply decision 
procedures as well, because they have the property that not only the 
premises imply the conclusion but also conversely. 


The rules governing quantifiers were given in Part I.1* 
*“S4. When the input problem contains quantifiers, the following preliminary 
simplifications are made: (i) All free variables are replaced by numbers, distinet 
numbers for distinct variables. (ii) Vacuous quantifiers, i.e., quantifiers whose 
variables do not occur in their scopes, are deleted. (iii) Different quantifiers are 
to get distinct variables; for example, if (x) occurs twice, one of its occurrences is 
replaced by (z), z being a new variable. This last step of modification is specially 
useful when occurrences of a same quantifier are eliminated more than once at 
different stages 
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The justification of the reduction to subproblems (Part I, T2.1) is 
obvious because all truth-functional rules are reversible and (x)(Gr & 
Hx) is a theorem if and only if (x)Gz and (x)Hzx both are. 

Usually T2.2 (Part I) is true, but restrictions are necessary, as the 
following example would show: 


(x)(Ey)[(z)Gyz & Hxy). 


Although x does not occur in the scope of (z), there is no way to bring 
(z) out of the scope of (x) because the variable y ties up the two clauses 
in the formula. There are several possible alternatives: one may make 
exact the restrictions needed, or record the scope of each quantifier in 
the usual manner, or use the easy simplification that when a quantifier 
governs a formula with two halves joined by a logical connective but the 
variable of the quantifier occurs only in one of the two halves, the 
scope is just that half. 

The test of connectedness of variables and functors (Part I, T2.3) is 
meant as a device to simplify the interconnections between quantifiers. 
In particular, the test gives a method for ascertaining that certain ap- 
parently complex sequents fall under the AF provability case. In order, 
however, actually to bring such a set of sequents into the AF form, we 
need in general transformations similar to those used in reducing a 
sequent to the miniscope form. Since the process can be tedious, one may 
prefer an alternative method of not carrying out the transformation but 
merely determining a bound k such that either the original sequent is a 
theorem or has a counter-model with no more than k& objects. If this 
alternative is chosen, a method for calculating the bound k has to be 
devised. 

In any case, when we have a finite set of atomic sequents and a set of 
governing relations among the variables and functors, we should further 
simplify the matrix, i.e., the set of atomic sequents by the familiar 
methods of dropping repetitions and immediate consequences. 

“$5. After the above preliminary simplifications, each problem is reduced to as 
many subproblems as possible in the following manner: (1) Eliminate in the usual 
manner every truth-functional connective which is not governed by any quanti 
fiers. (ii) Drop every initial positive quantifier (i.e., universal in the consequent 
or exister ‘al in the antecedent that is not in the scope of any other quantifier) 
and treat iis variable as free, i.e., replace all its occurrences by those of a new 
number. (i) and (ii) are repeated for as long as possible. As a final result of this 
step, each problem is reduced to a finite set of subproblems such that the problem 
is a theorem if and only if all the subproblems are. ; } 

12.1 The original problem is a theorem if and only if all its subproblems (in 
the above sense) are. : 3 ’ 

**T2.2 We can separate out Q and its scope from those quantifiers whose varia- 
bles do not occur in the scope of Q. 

“*T2.3 If two symbols, each a functor or a variable, are not connected in the 
final matrix, we can always so transform the original sequent as to separate the 
two quantifiers which give way to them.”’ 
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If there are two subsets of the set of atomic sequents which contain 
neither common variables nor common functors, then they can be 
separated. 

Moreover, each atomic formula that contains neither variables nor 
functors can be eliminated by the familiar method of replacing F(p) by 
F(t) & F(f). In other words, it can simply be dropped on the ground of 
the following consideration. E.g., take 


Guw,G11 — Gok. 
This is equivalent to the conjunction of: 


Guw,t — Gvk; 
Guv,f — Gok. 


But the second sequent is always true and can be dropped; the t in the 
first sequent can be dropped, so that we have 


Gu — Gok. 


After all the above steps, we arrive at a finite set of finite sets of 
atomic sequents which, taken together, are equivalent to the original 
problem. We may consider each finite set of atomic sequents separately 
and proceed according to the governing relations between their vari- 
ables and functors. 

We can view the set as a formula in the prenex form with a matrix 
in a conjunctive normal form. Or, if we prefer, we may replace — by +» 
and construe the variables as universal quantifiers, the functors as 
existential quantifiers. Then we get a negation of the formula in prenex 
form with a matrix in the disjunctive normal form. 

In either case, the remaining problem is to be handled by considera- 
tions such as those explained in Sections II through IV. 

There is an easily mechanizable procedure by which we can, in theory, 
not only prove all provable formulae, but also refute all formulae which 
have finite countermodels. All we have to do is test, besides the sequence 
M,, Mz, M;, ete., whether a formula is satisfiable in a domain with 
one object, or two objects, or etc. For example, given 


(x) (y)(Ez)Mayz, (1) 


if some of M112, M123, --- is contradictory, then the negation of (1) 
is a theorem; if relative to some finite domain, (1) can be satisfied, then 
the negation of (1) is not a theorem. For example, (1) is satisfiable in a 
domain with one object if and only if M111 issatisfiable; with two objects, 
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if and only if 


(x)(y)(Mayl v Mzy2) 


(x)[(Maxll v Mzx12) & (Mx21 v Mx22)| 
or 


(M111 v M112) & (M121 v M122)| 
& [((A7211 v M212) & (M221 v M222)| 


is satisfiable. 


VI. REMARKS ON MATHEMATICAL DISCIPLINES 


Besides the contrast between proving and calculating, there is a con- 
trast between symbol manipulation and number manipulation. There 
are problems such as proving trigonometric identities, factorization, 
differentiation and integration, which all appear to be mechanizable. 
In numerical calculations, it appears likely that the process of choosing 
one or another method of calculation can also be mechanized in many 
cases. 

There is the problem of applying the methods considered so far to 
deal with concrete examples. 

One example referred to in Part I' (p. 231) is Hintikka’s derivation 
of a contradiction from his own formal system." Here, intuitive under- 
standing is required to select from the set of all axioms suitable members 
which are sufficient to produce contradictions. _Experience, however, 
shows that, even aftera reasonable selection is made, to actually give an 
exact derivation of a contradiction remains quite a dreary affair. In 
such a case, the sort of procedure discussed in this paper can be useful. 

In fact, Hintikka uses five axioms to derive a contradiction. Write 
briefly: 


Hayz for ezeFakz He yk&zey &y Ez. 


The conjunction of the axioms is: 


(Ex)(Ey)(« # y) & 
(Ea)(Eb)(Ec)(Ed)(y) {fy # a D (y € a = (Ez)Hayz)| & 
ly #bD (y € b = ~(Ez)Hbyz)| & (2) 
y¥¥eD(yec=Hz=ly=avy=b))| « 
y~xdDyecd=y 
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The assertion is that (2) leads to a contradiction. In other words, (2) 
has no model, and its negation is a theorem of the predicate calculus. 
To decide whether this assertion is true, we only have to test (2) by 
essentially the method of Section III because (2) can be transformed into 
a formula with FA >2F prefix. Such a method yields also a proof or a refu- 
tation of the assertion that (2) gives a contradiction. 

In a different direction, we may consider some simple examples in the 
arithmetic of positive integers. 

First, we consider the example, «’ # x. We wish, in other words, to 
prove, with the help of induction, that this is a consequence of the 
axioms: 


, 


x #1, 
a’ #y'OurHr y. 


As a general principle, we try to use induction. Since there is only one 
variable, we reduce the problem to: 


(x)z’ € 1, (x) (y)(x’ = y’ Dx = y) - (3) 
(x)z’ # 1, (z)(y)\(2’ = y’ Dz = y), 2’ H#2—-2" £2’. (4) 


These can be dealt with by the program described in Part I, except 
that, to avoid confusion, we use now a, b, c, ete., instead of numerals to 


replace the positive variables. We have: 


These sequents are all true by substitution: 1 for z in the first two; a’ for 
u and a for v in the last two. 

As a somewhat more complex example, we take the commutativity of 
addition. In order to prove x + y = y + 2, we may use induction 
either on x or on y. We arbitrarily take the earliest variable: 


l+y=yt+1, 
rt+y=ytror’ty=ystr’. 


To prove 1 + y = y + 1, we make induction on y: 
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lI+1=141, 
lt+a=a+1-1l+aq =a’ +1. 


The first is a theorem by the property of equality. To prove the 
second, we use another general principle, viz., when a defined symbol 
occurs, we make use of the definition. In this particular case, we make 
use of the recursive definition of addition, and try to prove 


utl=w,utv =(ut+v),lt+a=at+l1lotlt+a =a’ +i. 


In order to derive the consequent from the antecedent, we start from 
1 + a’ and a’ + 1, use the equalities in the antecedent to transform 
them, and attempt to find a chain to join them. Thus, we may try to 
make all possible applications of the three equalities in the antecedent: 


(l+a)+1 (a+1)+1 
1+ a’——(1 + a)’ (a + 1)’ (a’)’ a’+] 
1 + (a + 1) 1+ (1+ a) 


a’ + 1lx——(a + 1) + 1-—(1 + a) + 1 
ay’ —(a + 1)’ (1+ a)’ l+a’ 


In general, we may begin two trees simultaneously from both sides of 
the equality, do not write down any term which has already occurred in 
the same tree, and stop when a common term appears on both trees. 
When we get to the more complicated situations, we have to investigate 
two additional things. First, it would take too long to search through 
trees, so that it is desirable to organize available informations in forms 
which are more quickly accessible. Second, we may exhaust two trees 
and still fail to get a common term. Then we need to prove some lemma 
which would join up the two trees. 

For example, the above graphs give us a proof of (5). To prove the 
other induction hypothesis, viz. (6), we may try to do the same with: 


utl=wrutr’=(utr),a+bdb=b+a--a+bd)=b)4a’, 
a+b (a+1)+b 
b + a<—(b + a)’ (a + 5)’ a+ b’ a+ (b+ 1) 
b+ (a+ 1) (b +a)+ 1 (a+b)+1 


In this way, we have exhausted the applicable cases of the equalities in 
the antecedent. Since we have proved the first induction hypothesis (5), 
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we can add it to the antecedent. Then we get some further extensions: 


(a + 1) + b>—(1 + a) + B, 
b+ (a+ 1)—% + (1 + a), 
a+ (b+ 1)\—~ + (14+ D). 


At this stage, we would ask whether any other given theorem can be 
used to join up the two trees for a’ + 6 and b + a’, or, if not, what a 
reasonable lemma would be. If the associative law has been proved, we 
may observe that the missing link is supplied by: 


(a+1)+bd=a+ (1+ 5). (7) 


Otherwise we should try to make a “reasonable” selection of some 
suitable lemma and prove it. If, for example, we have chosen (7), we 
would try to establish it by induction on a or on b. 

It is possible that the quantifier-free theory of positive integers, in- 
cluding arbitrary simple recursive definitions, can be handled mechan- 
ically with relative ease, and yield fairly interesting results. The re- 
striction to quantifier-free methods means that we are concerned only 
with quantifier-free theorems to be proved without using quantifiers in, 
e.g., applying the principle of mathematical induction. It is clear from 
works in the literature that this restricted domain of number theory is 
rather rich in content. It goes beyond logic in an essential way because 
of the availability of (quantifier-free) mathematical induction. 

With regard to the general questions of using machines to assist 
mathematical research, there is a fundamental contrast between problem 
and method. While it seems natural to choose first the objective (e.g., 
number theory or geometry) and then look for methods, it is likely that 
a more effective approach is to let the methods lead the way. For ex- 


ample, since the known interesting decidable classes of formulae of the 


predicate calculus either do not contain function symbols or do not 
contain quantifiers, we are led to the simple examples above: quantifier- 
free number theory or function-free set theory. 
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Prolate Spheroidal Wave Functions, 


Fourier Analysis and Uncertainty — I 


By D. SLEPIAN and H. O. POLLAK 


(Manuscript received August 1, 1960) 


A complete set of bandlimited functions is described which possesses the 
curtous property of being orthogonal over a given finite interval as well as 
over (— 2, ©). Properties of the functions are derived and several appli- 
cations to the representation of signals are made. 


I. INTRODUCTION 


It is pointed out in this paper that the eigenfunctions of the finite 
Fourier transform are certain prolate spheroidal wave functions. These 
eigenfunctions properly extended possess properties that make them 
ideally suited for the study of certain questions regarding the relation- 
ship between functions and their Fourier transforms. Here we shall 
study the fur: tions in some detail and present some applications to the 
representation of bandlimited functions. The property that we shall be 
most concerned with is the orthogonality of the functions over two dif- 
ferent intervals. The paper’ by Landau and Pollak which follows draws 
on this material, establishes other properties of the functions and pro- 
vides further examples of their application. 

After some definitions contained in the next section, we proceed to 
state without proof in Section III our main results. Certain applications 
of these results are then given in Section IV. The remaining sections of 
the paper are devoted to establishing the results already stated. 


Il, NOTATION 


In what follows, we denote by £,,° the class of all complex valued func- 
tions f(t) defined on the real line and integrable in absolute square. 
We adopt the notation 


4 


f(t) lla = if(t)| dt 
A 
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and refer to || f(t) ||,” as the total energy of f(t) and refer to || f(t) ||.4° 
as the energy of f(t) in the interval (— A,A). In an analogous manner, we 
denote by £,° the class of all complex valued functions f(t) defined for 
—A StS A and integrable in absolute square in the interval (—A,A). 

Functions in £,,° possess Fourier transforms. Upper and lower case 
versions of a letter will always denote a Fourier pair. We write, for ex- 
ample, 


, Fs yee ta) 
fo =s F(w)e'** de, (2) 
“aT J—c2 


00 
F(w) = f(t)e* dt. (3) 
— 20 
We refer to ¢ as time, w as angular frequency and w/2m as frequency. The 
functions F(w) are also integrable in absolute square. In this notation 
Parseval’s theorem is 


© eR 1 {* » \nm 
| f(t)g(t)dt = — [ F(w)G(w) dw. (4) 
og ; Qa J— 20 


We denote by @ the subclass of £,,° consisting of those functions, 
f(t), whose Fourier transforms, F(w), vanish if |w!| > Q. Here Q = 
2rW is a positive real number fixed throughout this paper. Every mem- 
ber, f(t), of ® can be written as a finite Fourier transform of a func- 
tion integrable in absolute square: 


Q 

: l Lee ie 

jit) = = F(w)e'* dw. (9) 
2m J-92 


Functions in @ are called bandlimited and ® will be referred to as the 
class of bandlimited functions. It follows from (5) that members of @ 
are entire functions of the complex variable ¢. 

From any function f(t) in £,° we can obtain a function, Bf(t), con- 
tained in @ by the rule 


Bf(t) = ms | F(w)e™™' dw, (6) 
2Tr J-92 

where F(w) is given by (3). We call Bf(t) the bandlimited version of 
f(t). We regard B as an operator whose effect on a function in £,’ is to 
produce its bandlimited version. In electrical engineering terms, Bf(t) 
results from passing f(t) through an ideal low-pass filter with angular 

cutoff frequency 2. 
We denote by D the subclass of functions, f(t), of £,° each of which 
vanishes for | ¢| > 7'/2. Here T is a positive real number fixed through- 
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out this paper. Members of ® are called timelimited and D will be re- 
ferred to as the class of timelimited functions. 
From any function f(t) in £,,” we can obtain a function Df(t) contained 
in D by the rule 
; “l<T/2 
Df(t) = sa bisa (7) 
0, |¢| > 7/2. 
We call Df(t) the tiémelimited version of f(t). We regard D as an operator 
whose effect on a function of £,,’ is to produce its timelimited version. 
We shall use the notation /(¢) € & to mean that the function f(t) 
belongs to the class ¥ of functions. 


Ill. RESULTS 


The statements made below are proved in Sections V and VI. 
Given any T > 0 and any 2 > 0, we can find a countably infinite set 
of real functions Yo(t),yi(t),wo(t),--- and a set of real positive numbers 


Wo >A > Ae > - > (8) 


with the following properties: 
i. The ¥;(t) are bandlimited, orthonormal on the real line and com- 

plete in @: 
(0 


’ 


Wi(t)yj(t) dt = * iit <r is oer. 


eee 


li. In the interval —7/2 S$ ¢ S 77/2, the y(t) are orthogonal and 
complete in £ 7/2": 
sell 0, tA] phe 
/ Vi (t)y;(t) at = 7 d t,7 = 0, Bain 6 x 
_r/2 \AG, ‘=J) 
ili. For all values of ¢, real or complex, 


T/2 + 
AWi(t) = | saast> tl vi(s)ds, 7«=0,1,2,--- 
Lr mt — 8) 
Further properties of the y’s are given in Sections V and VI. 

The notation used above conceals the fact that both the y’s and the 
\’s are functions of the product 27’. When it is necessary to make this 
dependence explicit, we write \; = A,(c), Wilt) = Pile,t), 7 = 0,1,2,---, 
where 2¢ = QT. 

Some values of \;(c) are given in Table I. It is to be noted that for a 
fixed value of ¢ the \; fall off to zero rapidly with increasing 7 once 7 has 





THE BELL SYSTEM TECHNICAL JOURNAL, JANUARY 1961 


TABLE I—VALUEs oF A,(c) = La(c) K 107? 


c= 1.0 c = 2.0 c= 4.0 


> 
- 


3.0969 .7258 
3. 5811 ‘ ). 2791 
3.9175 f . 2375 
.2114 8 9.2010 
.2714 3.7179 
4.6378 9.4914 
2.0413 .6716 
».5766 | S 2.1544 
.6183 2 2.1207 


. 8056 
.5564 
. 5868 
1522 
. 8882 
.9359 
3661 
.0489 
.7768 


9 9589 
9.1211 
5.1905 

.1021 
8.8279 
3.8129 

.0951 
2.2786 
3.6066 


.0000 
9.9988 
9.9700 
9.6055 
7.4790 
3.2028 
6.0784 
6.1263 
4.1825 


on 


— 


th WOW © 


Ue Det et et et 


— o 
+ 


° oe | 
wh fe ee ne 
_— 
= dD 
~I 


PE 


exceeded (2/2)c. (The significance of this will be discussed in detail in 
9 1 9 


a later paper.) Because of (9) and (10), namely | ¥; ||, = 1, || yi |! ro 
= \,;, a small value of \; implies that y(t) will have most of its energy 
outside the interval (—7'/2,7'/2) whereas a value of \; near 1 implies 
that ¥:(t) will be concentrated largely in (—7/2,7/2). This behavior 
of the ¥’s can be clearly seen in Figs. 1 through 5. Figs. 1 through 4 
show yYo(c,t), vile,t), we(e,t) and w3(c,t) for several different values of c. 
For c = 0.5, or (2/m)e = 0.3183, as shown on Fig. 1, Ys. and ys are prac- 
tically zero in the interval (—7'/2,7/2). For ¢ = 4, or (2/r)e = 2.546, 
as shown on Fig. 4, Yo is largely concentrated in the interval (—T7'/2, 
T /2). Fig. 5 compares Yo(c,t) for several different values of c. 


1V. SOME APPLICATIONS 


4.1 Extrapolation of a Bandlimited Function 


It is sometimes desired to extrapolate a bandlimited function known 
only on the interval (—7/2,7/2) to values outside this interval. Since 


any f © @ is an entire function, this extrapolation can be done exactly 


in principle. One could, for example, calculate successive derivatives of 
f at some point in ( —7'/2,7'/2) and form a Taylor series representation 
which would converge everywhere. In practice, however, such a Taylor 
series would necesserily be truncated and the resultant approximation 
to f(t) would be 2 polynomial which for sufficiently large values of 
|¢! would give a very poor approximation to f. This approximation is 
not, of course, bandlimited. 
The functions y; provide an alternative approach. Since f 

can write, from i., for all ¢ 


0 


f(t) > a,v,(t), 


0 
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where 


= | f(t)n(t) dt, 


: [ f(t)’ dt 


and the convergence in (12) is in the mean square sense 


co} N 2 
lim | i > | dt = 0. 
Now ¥—a@ 


Multiply (12) by y;(t), integrate and use (10). There results 


7/2 
a, = 2 f(t)yn(t) dt. (14) 


n %—T/2 
The coefficients in (12) can be determined by (14) from values of f(t) in 
the interval (—T/2,T/2). 
The above result suggests approximating f(t) for all t by 


N 


f(t) = > a, n(t) 


with the a, given by (14). The approximation (15) is itself bandlimited. 
The mean squared error is 


2 


(f(t) — fx(t)} dt = a (16) 
— 0 N+1 


and by (13) can be made as small as desired by making N sufficiently 
large. In the sense of (16), the extrapolation remains good for all ¢. 
The error in the fit of fy to f in (—T/2,T/2) is given by 


19 


T/2 ) 
| (f — fy) dt = Yar. (17) 
—T/2 N+1 

As the A, approach zero rapidly for sufficiently large n, it may happen 
that (17) is small for values of N for which (16) is still large. The fit of 
fw inside the interval should not be taken as an indication of the fit else- 
where. 


4.2 Approximation in an Interval by a Bandlimited Function 


Suppose now f(t) © Lr is known in the interval (—7'/2,7/2) but 
f is not necessarily a piece of a bandlimited unction. From i. above it 
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follows that f(t) may still be represented by (12) with a’s given by (14), 
but this representation is valid now only for |¢| < 7/2. If indeed f is 
not a piece of a bandlimited function, the series (12) will certainly not 
converge in mean square over the whole real line. 

The foregoing suggests the utility of finite sums of the form (15) as 
approximants to bandlimited functions having a prescribed form in the 
interval (—7'/2,7/2). The conditions of bandlimitation and prescribed 
form in (—7'/2,7'/2) are, of course, in general incompatible (unless in- 
deed, the prescribed form is a piece of a bandlimited function). How- 
ever, finite sums of the form (15) taken for all ¢ with a’s computed by 
(14) permit approximations by bandlimited functions to a prescribed 
f © Lr’. We are assured by ii. that the approximation can be made as 
good as desired in the sense that the right side of (17) approaches zero 
for large N. We have, however, 


oo N 
| fy (t) dt = Zz A, 
— OO 0 


N 
and, if f is not a piece of a bandlimited function, >> a,” grows without 
bound for increasing N. Thus, in approximating a piece of a nonband- 
limited function by a bandlimited function, we exchange goodness of 
fit in (—7/2,7/2) with wildness of behavior outside this interval. 
We now impose an energy restriction. Given f € L7/2. What g 
with prescribed energy || g |,” = E minimizes || f — g || r2°? Let 


f= Day,(t), |¢| s 7/2, 
g = > bw,(8), —-ax <ti<c ow, 
Then a simple argument gives 
AnXn 
ee’ 


where u is the unique positive number which satisfies 


b,, 


2 


| nn 


(u + An)?” 


If the constraint on g is that the energy outside (—T7/2,7/2) is pre- 


scribed, || g|\,° — |g /ln2e = EH’, rather than the total energy, the 
result is 


rs AnXn 
u(l — An) + An’ 
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where p» (again positive) is chosen to satisfy 


a 8 
E’ = . ¥ Gn An 


[u(1 sai An) + nl? 





4.3 Some Extremal Properties of Wo(t) 
7 


The w’s possess a number of interesting extremal properties. The most 
important of these, the fact that yp has the largest energy in (—7'/2,T/2) 
of all function in @ of unit total energy, is discussed in detail by Landau 
and Pollak.’ We comment here on two other extremal properties of yo . 

Let f(t) € &,” have total energy E = || f ||,”. The timelimited ver- 


sion of f(t) has total energy Ep = | Df \\,° = ||f\|r° S$ E. Since Df 
cannot be bandlimited, its Fourier transform has nonvanishing energy 
in |w| > Q@. The bandlimited version of Df, namely BDf, will therefore 
have total energy Ezn < En S FE. The operation BD transforms a 
member of £,,° into a member of ® with smaller total energy. Which 
members of £,” lose the smallest fraction of their energy under such a 
transformation? That is, for which f € £,” is » = || BD |\,"/|\ f ||," a 
maximum? 

The answer to this question, unique except for an arbitrary multi- 
plicative constant, is Dwo(t). This may be seen as follows. From (3), (6) 
and the definition (7) of D, ; 


Q e 7/2 


; Lo a : 
BDf(t) = | dw e'*' | ds f(s)e 
2a -Q J ? : 


= 7/2 


ws 


a T/2 


= | palt — s) f(s) ds, 
—7/2 


where we have written 


0 


sin Qr ie pie 
= =. | dw e". (19) 
WT Qn Ko 


Note that pe(z) is an even function of 7 and that from (19) and Parse- 
val’s theorem (4) it follows that 


« 
| polt =~ u) pol ees 8) du = palt —_ 8): (20) 
— 00 


Therefore, 
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0 


| BDf(u) |e [ du |(BDf(u)\[BDf(u)| 


~ 7/2 7/2 
| du | at | ds po(t — u)pe(u — s)f(t)f( s) (21) 
2 7/2 1/2 


T/2 7/2 4 
/ dt | ds po(t — s) f(t) f(s). 
Lrje sre 


Here we have used (20) and the fact that pe is real and even. 
Since from (21) we see that |! BDf ||,” depends only on values of f in 
(—T/2,T/2), it follows that u is equal to the maximum of 


T/2 T/2 
J 7 8 po(t — s) f(t) f(s) 
_pppgtt — Lon't Lon tO s) f(t) f(s 


IF llave 


over all f € Lr’. It is well known that the solution to this problem is 
v = Xo, where Apo is the largest eigenvalue of the integral equation 


r/2 
Af(t) = | po(t — s) f(s) ds, |¢] s 7/2, 
L_r/2 


and that v attains the value A» for f equal to a corresponding eigenfunce- 
tion. We shall see later that yp is such an eigenfunction. Thus f agrees 
with Yo in (—T/2,T/2) and so Dy is a function in £,” for which u 
attains its maximum value Xo . 

We now ask which f © @ as opposed to f € £,,” maximizes yu. That is, 
which bandlimited function loses the least (fractional) energy when first 
timelimited then bandlimited? The answer is Yo and the corresponding 
value of yu is Xo. 

To see this, introduce the representation (5) for f © @ into the nu- 
merator [as given by (21)] of uw. There results 


, ¥ 2 Q ; 
| BDf\i. = — | dw | dw’ I (w) F(w’ \K (ww), 
4r? Lo 2 
where we have set 
T/2 rz Me es 
K(ww) = | dt | dt pli —tje“a«™'. 
1/2 7/2 


To transform this expression further, introduce the representation (19) 
to obtain 
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1 Q T/2 7/2 
, U t(@—w” 4 —it’ (w’—w" 
K(ww) = = as” [ ane"? | eds 


2m Jo r/2 7/2 
Q 
; f ” / 
2r ! dw” prow — w” )prjo( —w) 
2 
=> 2rpr "a (ww i 
By Parseval’s theorem, (4), the denominator of » can be written as 
Q 
1 


2r Jo 


lf le = F(w) |? dw. 


Our task, then, is to maximize 





Q Q 
topic _ [5% [5 alone! PoP 


b= TCT — _ — — 


| f ||. Q 2 
: | | F(w) |° dw 
Lo 


over all F € Lg. The solution to this problem is h = yo, where yo is 
the largest eigenvalue of the integral equation 
Q 


, (2) / ’ , 
AF(w) = [ priz (ww )F(w ) dw . 
2 


Now prj” (w,w’) is the first iterate of prj2(w — w’). Therefore, yo is the 
square of the largest eigenvalue of the integral equation 


2 
AF(w) = | pri2lw — w )F(w ) dw . 


Q 


A change of variables reduces this equation to the form of (22) whence 
it is seen that w = Xo and that F(w) = Yo(w7'/2Q) for |w{.sS 2. From 
(29), which will be established later, it follows that f(t) = yYo(t). 


4.4 Problems Concerning Bandlimited Noise 


Much of the theory of detection, parameter estimation and predic- 
tion of signals in noise when observations are made in a finite time is 
based on the Karhunen-Loéve representation of the noise. (See Ref. 2 
for such a treatment of these problems.) This representation involves 
expansions in terms of the eigenfunction solutions of a certain integral 
equation. When the noise in question is second order stationary and with 
angular frequency spectral density uniform in (—Q,Q2) and zero else- 
where (bandlimited white noise), the integral equation in question is 
identical with (19), (22). The function y; and eigenvalues \; thus play 
an important role in numerous questions concerning bandlimited white 
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noise observed for a finite time. Their role in this connection has been 
pointed out previously in Ref. 3. 


V. THE PROLATE SPHEROIDAL WAVE FUNCTION 


The functions y;(c,t) are scaled versions of certain of the angular 
prolate spheroidal wave functions. A number of books*’'*”’ treat the 
prolate spheroidal wave functions in detail. We will draw freely from 
this literature. We adopt the notation* of Flammer.* 

When c is real, the differential equation 


Tu — 9 du 
dt? dt 
has continuous solutions in the closed ¢ interval [—1,1] only for certain 
discrete real positive values 0 < xo(c) < yxule) < xe(e) < --- of the 
parameter x. Corresponding to each eigenvalue x,(c), n = 0,1,2, 
there is a unique solution Sp,(c,t) such that So,(c,0) = P,(0) where 
P,,(t) is the nth Legendre polynomial. The functions So,(c,t) are called 
angular prolate spheroidal functions. They are real for real ¢, are continu- 
ous functions of c for c = 0, and can be extended to be entire functions 
of the complex variable ¢. They are orthogonal in (—1,1) and are com- 
plete in _ Son(c,t) has exactly n zeros in (—1,1), reduces to P,,(t) uni- 
formly in [—1,1] as c — 0, and is even or odd according as n is even or 
odd, n = 0,1,2, --- . The eigenvalues x,(c) are continuous functions 
of cand x,(0) = n(n + 1), n = 0,1,2, ---. 

A second set of solutions Ro,” (¢,t), n = 0,1, «++ , called radial prolate 
spheroidal functions, which differ from the angular functions only by a 
real scale factor, 


(1 — t’) +(x - ct )u = ( (23) 


ls \ Y \ 
Ron (c,t) = kn(c)Son(c,t), 
are of use in many applications. These radial functions are normalized 
so that 


via ] 
Ron’ (¢,t) > 3 008 [ct — 4(n + 1)a] 
i 
ast— o, 
The equations 


2 ee ee ‘sin c(t — 8) ; 
. | (c,1 "Son ( ct) . | po — Son c,s) ds, (24 ) 
T Ly w(t — 8) 


1 
21” Ron’ (€,1)Son(et) = [ ce" So, (es) ds n=0,1,2,--- (25) 
1 


* The reader should be cautioned that various authors disagree not only on 
notation for these functions, but also in their method of normalization. 
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are both special cases of more general integral relations satisfied by 
prolate spheroidal functions that can be found in the literature. They 
are valid for all ¢, real or complex. 

Equation (24) shows that Spo,(¢,t) is a solution of the integral equa- 
tion 


Af(t) = | p(t — s)f(s) ds, [¢| <1 
J} 


corresponding to the eigenvalue 


Dp 

a 1 2 - 
ra(c) = — (Re (e,1)F, n= 0,1,2,---. (27) 

Tv 


Here p.(7) is given by (19). Indeed, the completeness of the So, in £:° 
assures us that the quantities (27) are the only eigenvalues of (26) and 
that if these quantities are distinct, the So, are (apart from multipli- 
cative constants) the unique £;° solutions of (26). If several of the quan- 
tities (27) are equal for different values of n, then linear combinations 
of the corresponding So, will also satisfy (26). Within the sense of this 
degeneracy, then, the Sp, are unique solutions of (26). In Section VI 
we shall see, indeed, that this degeneracy does not occur. 

Equation (19) and Bochner’s theorem (Ref. 8, Theorem 23, p. 95) 
show that the kernel of (26) is positive definite. The quantities (27) 
are therefore strictly positive. Set 

al 


[un(e)? = J [So(cp)F ae 
-1 


We now finally define 


A/ Kal) 
u,(C) 


vn (c,t) Son(c,2t/T ). (28) 
Properties ii. of Section III now follow directly from definitions and 
the orthonormality and completeness of the So, in (—1,1). 
A change of variables and the definitions (27) and (28) convert (24) 
into (11). A change of variables converts (25) into 
n 1 e 2 
1 QR (c,1) I ia - 
v,(c,t) ey, (ewT /22) dw, (29) 
v 2nr Jo 
which shows y,, . Indeed, since the function ,(¢c,w7'/2Q) are com- 
plete in —2 S w 0, Parseval’s theorem shows that the y,(t) are 
complete in ®. The remaining assertion of i. of Section III, namely (9), 
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follows from a computation. From (11) we have 


| dt p;(t)y;(t) 
~T/2 


2 
| 2 T/2 
= at [ ds po(t — s)yW;(s) | du polt — u)yp;(u) 
Aid; ae — 7/2 lo 


—Ti2 


1 T/2 T/2 od 
= | du | ds ;(s)p;(u) | dt po(u — t)pel(t — s) 
AiAj Lr/2 7/2 = 


| i [ 2 
= - du W;(u) ds plu — s)W;(s) 
AiAj 7/2 vi L7r/2 . v 


1 T/2 
= du p;(w)ypi(w) 
Aj rie 
Here we have used (20) and (10). 
All properties of the y’s asserted in Section III have now been estab- 
lished except for (8). To this end we devote the next section.* 


VI. NONDEGENERACY AND ORDERING OF THE EIGENVALUES OF (26) 


We have seen that the So,(c,t) are solutions of (26) with eigenvalues 
given by (27). We show now that we cannot have two distinct Sp, 
belonging to the same eigenvalue X if ¢ > 0. 

Let fi(t) and fo(t) be two linearly independent solutions of (26) for 
the same A, c * 0. Then 


ol 


Af(t) = | p(t — s)fi(s) ds, 
3 


1 
Afi (t) = | p(t — s)fi(s) ds, 
1 


1 
AFT (t) = l p(t — s)fi(s) ds, 


=} 


1 
Afe(t) = [ p(t — 8 )fo(s) ds 


1 


* Ville and Bouzitat® recognized (independently of the earlier Ref. 3) that the 
solutions of the integral equation (11) are prolate spheroidal functions. They 
assert that the eigenvalues \, are ordered as in (8) when y, is identified with So, 
but no proof of this fact appears in their paper or apparently elsewhere in the 
literature. 
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and 
1 
PLA ‘ ” . 
Afe(t) = | p.(t — s)fo(s) ds. 
a 


Assume now that f; is even and fe is odd. Integrate (31) by parts to 
obtain 


1 
Afi (t) = fi(1)[p.(—1 — t) — p(1 — t)] + | p(t — s)fi(s) ds. 
LH} 


Multiply this equation by f2(¢) and integrate to obtain 


1 


rX | felt) filt) dt = AC) [fe(—1) — f2(1)] 


ant. 
1 1 : 
+/ at | ds p(t — s)fi(s)fo(t). 
—} — I 


Now multiply (33) by fil t), integrate and subtract the result from (35). 
One finds Afi(t)[fe(—1) — fe(1 )} = 0, or 


fi(A)fe(1) = 0, fi even, fe odd. (36) 


Assume now that f(t) and fo(t) are of the same parity, i.e., both even 
or both odd. Multiply (32) by fe(t), multiply (34) by fi(t), subtract 
and integrate. There results 


1 


1 
rx | dt(ftfe —fth) => | > iks ~ 8K) 
hm dt 


ey 


2M fi(Dfe(1) — fe(1)fi1)] = 0 


fil fa ( 1) = j2(1 fi 1), f; and fz of same parity. (37) 


For any two linearly independent solutions of (26) belonging to the 
same eigenvalue we must have either (36) or (37) hold. But we shall 
show that both of these conditions are impossible for two different S 
functions, say So,(c,t) and Som(c,t). From the differential equation (23), 
we see that 


2Son(1) = (xn — €°)Son(1). (38) 


If So,(1) vanishes, then so does So,(1). But differentiating (23) shows 
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that if So,(1) and Son( 1) vanish so does So, (1). Repeated differentiation 
(which is possible since the So, are entire) shows that if Sp,(1) = 0, 
then So,(t) = 0. Therefore condition (36) cannot hold. On the other 
hand, since So,(1) # 0, Som(1) # 0, (37) can be written 


Som(1) _ Son (1) 


Som(1)  Soa(1) 


(39) 


from (38). However, it is known that the eigenvalues of the differential 
equation (23) are nondegenerate if c is real, so that (39) cannot hold if 
m # n. The eigenvalues (27) are thus seen to be distinct. 

By their definition, the So, functions are indexed so that the eigen- 
values of the differential equation (23) xo < x1 < x2 < --- are mono- 
tone increasing functions of their index. We have defined y, in terms of 
the So, by (28) and have labeled the corresponding eigenvalue of (26) 
\, by (27). There remains the task of proving that the X\, are ordered as 
in (8). 

Our argument makes use of the fact (just demonstrated) that for all 
real c ¥ 0 the \,(c) are nondegenerate and the fact (see for example 
Ref. 10, vol. I, p. 128) that the eigenfunctions and eigenvalues of (26) 
are continuous functions of its kernel. Thus if we can prove that for 
some c > 0, 


do(c) > Au(c) > Ao(c) He 65 
then continuity and nondegeneracy of the \’s allows us to assert this 
ordering for all positive c. 


We now establish this ordering for c sufficiently near zero. Let y, and 
Wn be successive eigenfunctions of (26), ¢ ~ 0. Then 


1 
AnWa (t) = [ p(t — s)y,(s) ds, 
1 


AnstWaai(t) = | p(t — 8)Wnai(s) ds. 
1 


Multiply the first of these equations by Aj 4:%n41(t), multiply the second 
by AnWn(t), add the results and integrate to obtain 





62 THE BELL SYSTEM TECHNICAL JOURNAL, JANUARY 1961 
1 
, , 
AnAnd1 [ (Wan +1 + Vn iW» ) dt 
v1 


al +1 
= An+1 | 


dt ds p(t — 8)Wnsi(t)yn(s) 
J} J} 


1 1 
+ a | at | ds p(t — 8)n(tWn4i(8) 
= I 
‘a 1 
= (rA, - vst) | at | ds p-(t ~ 8)Wn(t)Wniils) 
=" —1 


1 
= (An — wee | W,(t)Wrar(t) dt 
=} 


l 

Winer at 
1+ 
|] VrWnar at 
=} 


d 


+ . , 
Now asc > 0, y, — P,(t), the nth Legendre polynomial, and y, — 
, . . . . . 

P,,(t). The denominator of the fraction in (40) approaches 


1 l 1 
| P,P... dt = PF xs = / Pu? dt = 2 
-1 1 1 


since the integral on the right vanishes and P,,(1) = 1. The numerator 
approaches 


al 
| P*P,., dt = 0. 
—] 


By making c sufficiently small, therefore, the fraction on the right of 
(40) is of absolute value less than unity and A, — Ana 
A,{L + O(1)] 2 O. Since for ¢ ¥ 0 the X,, are all distinct and positive, 
the ordering (8) must hold. The limiting eigenvalues for ¢ — 0 are 
OQ = r+» = A = ho 


VII. COMMENTS 


It is worth pointing out that the basic importance of the y, for the 
study of the relation between functions and their Fourier transforms 
stems from (25), which shows that the So, are eigenfunctions of the finite 
Fourier transform kernel. Indeed, many of the important properties of 
the y’s (i. and ii. of Section III, for example) follow directly from (25) 
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or its first iterate (24), without explicit use of (23) or recognition of the 
So, as angular prolate spheroidal wave functions. 

In the interests of simplicity of presentation, we have not put forth 
the theme of this work in its most general form. We here make just one 
comment in this direction and leave other generalizations to the in- 
terested reader. The curious orthogonality over two different pointsets 
of the analytically continued solution of (22) will hold whenever (20) 
is true and the solutions are in £,”. For example, if the kernel p(r) of 
(22) is even and has a Fourier transform constant on intervals and zero 
elsewhere, e.g., pi(t) = pelt) cos ar, a > Q, then the double ortho- 
gonality maintains. The eigenfunctions for the bandpass kernel p;(7 


do not seem to be expressible in terms of well-studied functions. Com- 
putations in this case indicate the existence of degenerate eigenvalues. 
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Prolate Spheroidal Wave Functions, 
Fourier Analysis and Uncertainty —II 


By H. J. LANDAU and H. O. POLLAK 


(Manuscript received July 6, 1960) 


The theory developed in the preceding paper’ is applied to a number of 
questions about timelimited and bandlimited signals. In particular, if a 
finite-energy signal is given, the possible proportions of its energy in-a 
finite time interval and a finite frequency band are found, as well as the 
signals which do the best job of simultaneous time and frequency concen- 
tration. 


I. INTRODUCTION AND SUMMARY 


It is a common experience in the communications field that one can- 
not simultaneously confine a function f(t) and its Fourier transform 
F(w) too severely. The most familiar statement of this phenomenon is 
the Heisenberg uncertainty principle: If we measure the time-spread T 
of f(t) by 


| (t — to)* | f(t) Prat 


“——<@D 


[iso |? de 


7? - 


and the frequency-spread Q of F(w) by 
(w — wo)” | F(w) |? dw 
0 a SS. — 
| | F(w) |? dw 


then, for any choice of f and w), QT = 3. Thus T and Q cannot, for 
any Fourier transform pair, be both small. Equality will hold if f(¢) 
[and hence F(w)] are gaussian, and f) and wy are chosen as the means 
of | f(t) |? and | F(w) |’ (in this case both zero). This result, while 


65 
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demonstrating that our experience with timelimiting and bandlimiting 
is indeed related to mathematical truth, does not succeed in providing 
a very good understanding of what is really happening. We should like 
to know just how close one can come to simultaneous limiting in both 
time and frequency, and what the price is that one has to pay. We need 
a sharper measure of the concentrations of f(t) and F(w) than that 
afforded by the above variances of | f(t) |’ and | F(w) |’, a measure 
which, if possible, will depend on the behavior of f(t) in a given finite 
time interval, and of F(w) in a given finite frequency band. 

An early attempt to meet this need was made by L. A. MacColl, 
who around 1940 proved the following previously unpublished form 
of the uncertainty principle: 


If 
tot+T ? 
/ | f(t) [dt 


0 


| | f(t) |? de 


= di 


and 


° 


/ ) | F(w) | dw 


w( 


| | F(w) | dw 


QT > 2rayjae. (2) 


This theorem does indeed emphasize the behavior of f(¢) and F(w) in 
given finite intervals. The quantity a , representing the proportion 
of the total energy of f(t) which is in the time-interval (fo) , 46 + 7’), is 
especially satisfying as a measure of the spread of f(t); on the other 
hand, a2 has no immediate physical interpretation. A further difficulty 
with (1) is that there are no functions for which equality can be 
achieved, although in practice the estimate is quite good. 

A more useful form of the uncertainty principle would replace the 
above measure a2 by the proportion of energy of F(w) in a frequency 
band, that is, by a definition similar to that of a; . This is done in the 
present paper. We shall see that if 
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tot+T/2 ; 
[iso Pat 
t 


o—T/2 


[sm Pa 


2 
| | F(w) |° dw 
Lo 
| | F(w) |? dw 


then 
QT = (a,B), 


where ©(a,8) will be found explicitly, the inequality will be sharp and 
functions yielding equality will be given. The optimal functions f(t) 
will always be real if, as in the above statement, the frequency band 
is centered at zero. The same inequality holds if the frequency band 
under study is not centered at zero, but then the optimal functions are, 
in general, complex-valued. 

The simplest special case of our result arises if 8 = 1, so that all 
of F(w) is contained in |w| S Q, and F(w) = 0 for |w| > & The 
question “if @ is given, what is the minimum 7°?” can now be re- 
phrased “if Q7 is given, what is the maximum @?”’ Let us introduce 
the following notation: The square norm of f is the total energy of f: 


Wit = [140 Pat. 


Timelimiting a function f produces a function Df which is f restricted 
toj¢| s at /2: 
ee | MR Bo os 
Df = ¢: ele am 

. 0 Wjé| > T/% 
Bandlimiting a function f produces a function Bf whose Fourier trans- 
form agrees with the Fourier transform of f for | w| < Q, and vanishes 
for|w| > Q: 

1 Q 
ale 


a7 4-2 


F(w)e dw. 
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By writing 


Fiw) = f(s)e™ ds, 


ha 
we see that an alternative expression for Bf is given by 


gets —s8 
Bf=-]| f(s) ain AE — 8) 
T Jax t—s 
It was shown in the preceding paper’ that if a function is band- 
limited and then timelimited its energy must be reduced by at least a 
factor Xo , where Ap it the largest eigenvalue of the integral equation 
Be ie sin Q(t — s) 


f (s) ————_——— ds. (2) 
w /—rT/2 t—<s 


Af(t) 


If, in particular, a function is already bandlimited (f = Bf), then by 
this result || Df ||? < >. This, now, is just the special case of the 
uncertainty principle which we have been seeking: If 8 = 1, then 
asvVvo. 

In the sequel, we shall take a longer look at this formula and its 
significance; let us, however, state the full result for all values of a 
and g: 

Theorem: There is a function f such that || f || = 1, || Df || = @ and 

Bf || = 8, under the following conditions, and only under the follow- 
ing conditions: 


l. Ifa = 0, when 0 < 8 1. 


2160<a<SYr, whend0S 6S 1. 
3. If-VYd Sa <1, when cos | a + cos 8B = cos WXo- 
4. Ifa = 1, when 0 < B S Vd. 


The body of the present paper will cover the following sequence of 
topics: Section II will develop the properties of timelimited and band- 
limited functions, and the geometric interpretation of these properties, 
which we require. Section III contains the proof of the quoted theorem, 
a discussion of the “‘best”’ functions, and a number of pertinent graphs 
and numerical examples. Section IV indicates possible extensions of the 
theory, and includes the interesting result that if a timelimited function 
d and a bandlimited function b are given, it is always possible to find a 
“smallest”? function f so that Df = d and Bf = b. Finally, Section V 
gives applications of the preceding theory to filter theory, data trans- 
mission and antenna theory. 
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II. SPACES OF TIMELIMITED FUNCTIONS AND BANDLIMITED FUNCTIONS 


We are concerned, in the present paper, with the collection of func- 
tions f(t) which are square-integrable on (—%©,«). These form a 
Hilbert space, denoted by £’, in which the inner product (f,g) is de- 
fined by 


(fo) = [OGD at, 


and || f ||? = (f,f) as usual. 

The collection of timelimited functions forms a linear subspace D 
of £’ so that if f, and f2 are timelimited, so is af; + bf: . Furthermore, D 
is complete, which means that if we have a sequence of functions {f,}, 
fa € Dand if || f. — fm || 0, then there is a function f € D such that 

Exactly the same statements may be made about bandlimited func- 
tions; they form a complete linear subspace ® of £”. The latter state- 
ment follows from the earlier one through the Parseval relation for 
Fourier transforms: If F and G are the Fourier transforms of f and g 
respectively, then 


2 


—_ eo eae 
f(t)g(t) dt = — F(w)G(w) dw. 
— 20 g 2r — a0 


We shall call two functions f and g orthogonal if 
(f,g) = 0. 


Notice that Df and f — Df are orthogonal, since each one vanishes 
where the other one does not; by the Parseval relation, Bf and f — Bf 
are also orthogonal. 

The inner product permits us to define the angle between two func- 
tions f and g as follows: By the Schwarz inequality, we know that 


lols isll-lolls 
since 

| Re(fg) | S| (fg) |, 
we know that 


iw Re(f.g) “s. 
= al ry. g | = 


We may thus define the angle 6(f,g) between the functions f and g by 
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6(fg) = cos” _Re(fg) 
ey fil-ilg 
The extreme values 0 and x for @(f,g) can be reached only if f and g 
are proportional (so that equality holds in the Schwarz inequality) 
and (f,g) is real. 

Suppose now that f © @ and g € QO, and that neither function 
vanishes identically. What can we say about the angle between them? 
The angle can vanish only if for some constant k, f = kg. But since ® 
and D are linear spaces, this would mean that f is both timelimited and 
bandlimited, and this is known to be impossible.ft If, then, the angle 
cannot vanish, can it be arbitrarily small? This is the key question 
which shall occupy us for some time. Let us consider, first of all, a 
fixed function f € ®, and an arbitrary g © D. We know that 6(f,g) can- 
not vanish; is 6(f,g) bounded away from zero? If there is a greatest 
lower bound for 6(f,g), is it assumed for some particular functions 
g © D? In this case, the answers are quite simple, and are given by the 
following: 

Lemma 1: If f © @ is given, then 
inf 6(f,g) > 0. 


3) 


This infimum equals 


1 || Df || 
f 


cos 


and is assumed by g = kDf for any positive constant k. 
Proof: If g is any function in D, then 
Re(fig) = | (fg) (Df,g) 
since 
f — Df + Df and (f — Dfg) = 0. 
But 
(Df,g) | 


+ For then 


F(w) e** dw, 
-2 


since f € &, would be an analytic function of the complex variable ¢ whose 
vanishing for | ¢ | > 7’ would imply f=0. 
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so that 


Re(fg) = || Df|| _ _Re(S,Df) 


Tl Vol = Wel ~ WF Vy 
Since cos @ is monotone decreasing in (0,7), it follows that 


O(f.g) = O(f,Df) 


for any g © ®, with equality whenever g and Df are proportional. This 
proves the lemma. 

We proceed now to the case of arbitrary f € ® and g € 9D. Let us 
say, for convenience, if 

inf 6(fg) 

7S 
is actually assumed by specific functions, that the spaces ® and D form 
a least angle. We now have the following: 

Theorem 1: There exists a least angle between @ and D. This angle 
equals cos ' 4/X»), and is assumed by Yo € ® and Dy © D, where Xo 
is the largest eigenvalue of (2), and Y the corresponding eigenfunc- 
tion. 

Proof: By the preceding lemma, 


min 6(f,g) = cos” DS | , 
g€D WF ll 


so that 


Df || 


SEQ SER iJ 
gED 


inf 0(fg) = inf cos 


and the infimum on the left of (3) will actually be assumed if the in- 
fimum on the right is. It was shown in the preceding paper’ that any 
f © @® may be expanded in a series, convergent in L’ mean, of the 
eigenfunctions y, of (2), 


Then 
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since 


Df = >> a,Dy,, 


it follows from the properties of {Dy,} that 
Df |? = 2 | an [Aa . 
Thus 


1! 1 : 3 ; 
cos Dy '= cos’ (4 “- | 2) ‘ 
De | an | 


f 
4° ° ° ° 1 ° 
Since it was shown in the preceding paper that A, < Ao, if n = 1, 


it follows that 
(% | a, |” ) 
max ‘ 
p 


is achieved if a, = 0 for n 2 1, so that the minimum possible value of 

1 | Df | 
if ? 

namely cos ' ~/Xo, is actually assumed if f = yo, and g = Dy. The 

theorem is proved. 

We have thus found that the two subspaces ® and D of £’, which 
have no functions except 0 in common, actually have a minimum angle 
between them, so that, in fact, a timelimited function and a band- 
limited function cannot even be very close together. With the aid of 
this result, as we shall see, the uncertainty principle which we are 
seeking will follow. 

In preparation for the coming theorems, we must consider one 
further aspect of the spaces ® and D. How close do ® and D together 
come to filling up all of £°? The two specific questions which concern 
us are the following: (i) if {f,}, f. = bn + d, is a Cauchy sequencet of 
functions in ® + D, what can the limiting function f look like; and (ii) 
do there exist functions f € £* orthogonal to both ® and ® (i.e., to 
every function in ® and )? The answers to these questions are the 


cos 


subjects of the subsequent two lemmas. 
Lemma 2: If {f,} is a Cauchy sequence of functions of the form f, = 
t A Cauchy sequence of functions is a sequence such that || f, — fm || + 0, so 


that, by the completeness of Hilbert space, there exists a limiting function f 
such that || f — fn || - 0. 
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d, + b, where d, € D and b, € @ for each n, then the limiting function 
f is itself of the form d + b, where d © Dand b € B&B. 
Proof: For each f, = d, + b, , we may also write 


fn = (Dn — Dbn) + (Dba + dn). 


Here Db, + dn € D, while b, — Db, 1 D. It now follows from the 
fact that the f, form a Cauchy sequence that the functions b, — Db, 
do; for 


| bn — Dbn — (bm — Dbm) ||? + || Don + dn + Dom + dy |I’, 


so that 

| bn — Dbn — (bm — Dbm) || S& || fn — Sm || . 
But now, since {b, — Db,} forms a Cauchy sequence, so does {b,} itself. 
For 


9 


ll bn — Dm ||’ = || Dib, — b.) |’ + | (bd. — 62) — Did, — b.) |’ 


b 


and by Lemma 1, 


| D(b, — bm) || S Ado || bn — Om | 


, 


so that 


2 e b, — Db, — (bn — Db») 4 


ll Dn — Om | 
\] a m 1 <a a 
Since {b,} is now a Cauchy sequence, there is a function b © @ such 
that 


'b — b, || 3 0. 


Thus {f,} and {b,} both converge in norm, and hence so does }d,,}, and 
to a limiting function d © D for which 


f=b+d. 


We have thus shown that taking a limit of sums of functions from 
® and D gives us nothing new, but only, once again, a sum of functions 
in ® and . We may abbreviate this by saying simply that ® + ® is 
closed. 

Lemma 8: There are infinitely many functions in £* which are orthog- 
onal to ® + D. 





74 THE BELL SYSTEM TECHNICAL JOURNAL, JANUARY 1961 


Proof: The functions 


| f7T/2+nsjt| S$ T/2+n+1 


n = 0,1, 2,--- 
elsewhere . 


ee 

are instances of functions not in ® + QD, since the portion of f, in 
|t| > 7/2 is not a piece of a bandlimited function. Lemma 2 permits 
us to write the best approximation to f, from @® + ®D in the form 
b, +d, , where b, € @® and d, € QD; then 


; Ne _ i. == Dy bao d,, 


are distinct functions in £* which are orthogonal to ® + 9. 

There are in fact, in some sense ‘‘many more” functions in £° — ® — 
® than in @® + D; we do not know, however, of any really convenient 
representation for such functions. 


Ill. THE UNCERTAINTY PRINCIPLE 


We begin by restating the theorem announced in Section I. 

Theorem 2: There is a function f € £* such that || f || = 1, |! Df || =a 
and || Bf || = 8, under the following conditions, and only under the 
following conditions: 


l. fae = 0, when 0 < 


B 
if0O<a< Vr, when 0 < B 


IfVr Sa <li, when cos ‘a cos 'B = cos 'V/Xo. 
If a i. when 0 < 8 Vo. 

Proof: Let G be the family of functions f € £° with 

Df | = a, and let us, for each case of a, determine 


sup 8 = sup || Bf 


JEG JEG 


We shall also show, in each case, that any value of 8 less than the 
supremum can be realized by an appropriate function. Whether or 
not the supremum itself can be realized will vary from case to case. 

Case 1. a = 0. If a = 0, the family G can contain no function with 
8 = 1. For if f © G with 8 = 1 we must have f © @, whence f is ana- 
lytic and vanishes for | | < 7'/2 only if f = 0. This is a contradiction. 

To show that G contains functions with values of 8 arbitrarily close 
to 1 we set 


Vr or Dvn 


f* — 


Vi — >,’ 
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where \,, and y, are respectively an eigenvalue and corresponding eigen- 
function of (2). We observe that f* € G and that 6 = | Bf* |! = 
V1 — X,. Since there exist eigenvalues \, arbitrarily small, there 
exist functions in G with values of 8 arbitrarily close to 1. 

To find functions in G with values of 8 between those already covered, 
we consider e’’f*(t), which belongs to ¢ since || e'‘f* || = || f* || = 1 
and || D e’*'f* || = || Df* || = a. For B we find 


, 
2 


ee 
6 = ||Be''f = [ | F*(w) |? dw> 
“—p—2 
where /* is the Fourier transform of f*. This quantity is continuous in 
p and approaches zero as p > ©, since F* € £°; thus S contains func- 
tions with all smaller values of 8, except possibly 8 = 0. 

A function f in G for which 8 = O must have the property that 
Df = Bf = 0; the existence of such functions was demonstrated in 
Lemma 3. 

This completes the proof in Case 1; if we reverse B and D in the 
preceding arguments, we find that 8 = 0 is possible if and only if 
0 < a < 1; thus the minimum 8 in Cases 2 and 3 has also been estab- 
lished. 

Case 2.0 <a < Vd». Since \, ~Oasn— &, weean find an elgen- 
value \,, < a. Let y, be the corresponding eigenfunction, and consider 


f* aS V/ a —_ As Wo + V/ Xo — a Wn 
: V Xo — rr 





(oO) 


We have f* © @, and || f* | = | Bf* | = 1, while a simple computation 
shows that || Df* || = a. This, then, covers the case 8 = 1; by picking 
e’’'f*, as in Case 1, we may obtain any 0 < 6 < 1, and 8 = O is covered 
by the remark immediately preceding Case 2. 
Cases 3 and 4.» S a S 1. Fora function f © G, let us find the 
closest point to f on the plane spanned by Df and Bf; we then can 

write 
f = XDf + wBf + 9g, (6) 


with g orthogonal to both Df and Bf. Taking the inner product of (6) 
successively with f, Df, Bf and g, and using the fact that f © GS, we 
obtain 


ha + us + (9,f), 

ha’ + u(Bf,Df), 

\(Df,Bf) + up’, 
(fg) = (9.9). 
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By eliminating (g,f), \ and » from the above equations we find, for 
aB ~ 0, 
_ | (Df,Bf) - 


8° — 2 Re (Df,Bf) = —a’ + (1 ——a 
ar i (7) 
3 


7 r( ee) 
— g 1 a . 


a8? 
We next set 

(DIB) 
Df ||-|| Bf 


The angle @ is that formed between Df © D and Bf © @B so that, by 
Theorem 1, 


Re = cos 0. 


62> cos’ Vr. (8) 
Since 
a8 cos 6 = Re( Df, Bf) s | (Df, Bf) | s 
we have 


cats uw (Df,Bf) | < 1— cos @. (9) 


—_ a8? _ 


Introducing @ into (7), completing the square on the left-hand side, 
and applying (9) we obtain 


(8 — acos 6)’ < (1 — a’) sin’s, (10) 


with equality if and only if g = 0 and (Df,Bf) is real. From (10) we 
find immediately 


1 
<= cos(@ — cos a), 


whence by (8) 


1 1 
B = cos(cos V Xo — COS a), 


cos ‘a + cos '8 = cos a/ Xe. 
Equality in (11) is attained for the function 
f* = pyo + qDW , 
with 


1 — a’ 


_—— 1 — Xe 
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ie. /1 — a 
"athe Ft 
since f* satisfies all the conditions for equality in the above sequence 
of inequalities; the constants p and gq are chosen so that f € G. As in 
Case 1, all smaller values of 8, except possibly for 8 = 0, are attainable 
by the functions e’“f*(t) with suitable values of p, and, by the argu- 
ment above, the family G contains functions with 8 = 0 as well, except 
when cos ‘a = 0. Thus, in Case 3, G is made up of functions for which 
B takes on all values for which 


cos ‘a + cos ‘8 = cos : a/ de. 
If, however, a = 1, we must exclude 8 = 0, so that we obtain in Case 4 
0<B< Vn. 


The result of Theorem 2 is illustrated in Fig. i, which shows the 
permissible region in the (a’,8°) plane for various values of ¢ = Q7'/2 


atT= 


9 
MN P6.25 








a? 
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az 


Fig. 1 — Possible combinations of a? and 6? for different Q7. 
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For each value of c, this region is bounded by the line segments 


= Q for < i 


A 


= 0 for < i 


IA A 


> 


for 


for 


IIA 
d 


(2), 


and the curve cos ‘a + cos” OS WV dole), which is labeled by 
the appropriate value of c. 

An interesting phenomenon is brought up by the line a& + 6 = 1, 
which is labeled with c = 0. This labeling agrees with Theorem 2 in 
the following way: 

Ifa + 8 < 1, then cos ‘a + cos 'B = x/2, which automatically 
exceeds cos | ~/Xo for any c, no matter how small. In physical terms, 
this observation states that if the proportions of energy of f(t) in 

t| s T/2, and of F(w) in |w! S Q, add up to less than the total 

energy of f(t), then we have really put no restraint on 2 and 7, and 
an arbitrarily small Q7' product will still permit this distribution of 
energy. It is only when a + 8 > 1, so that the energies in | t| < 
T/2 and in |w| < Q add up to more than the total energy, that a 
nonzero lower bound on 7 is implied. 

Fig. 2 gives a detailed plot of what is essentially the top (or the 
right) edge of Fig. 1. We plot do(c), the maximum of a if 8 = 1, 
against c. We note that A»(¢) — 1 quite rapidly as c — ~ ; the approach 
is exponential, but the exact rate has not been proved. Fig. 2 also gives, 


1.0 r -— 


ooeger= 





-- 
--" | 


MAXIMUM ~~baa 


< 





+ ++ + 


sin nt | 
—S 


FoR f(t)=c 











aT 


Fig. 2 Possible a? if 8? = 1. 
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for comparison, the proportion of energy in | ¢| < 7'/2 for the function 


sin Q 
fit) = = t , 
t 


which has sometimes been “‘intuitively”’ considered as the bandlimited 
function which is as concentrated in time as possible. For small Xo , 
it appears, f(t) is indeed essentially as good as the optimal function; 
if, however, we wish to achieve a proportion of energy like 92 per cent, 
we see that Q7' = 4.5 suffices, while use of (sin Qt) /t would require 
QT = 8.5. For a proportion of 99 per cent, the minimal Q7 is 6.25, 
while (sin Q¢)/t would require a value of QT of about 30. 

Let us consider one more numerical example. If values of a° = 0.977 and 
8° = 0.96 are desired, what are the minimum 77, and the corresponding 
optimal function? From cos ‘a + cos 'B = cos’ WXo we find A» = 
0.88, so that OT = 4, orc = 2. If, now, Wo(t) is the first eigenfunction 
corresponding to ¢ = 2, then, by (12) and (13), the optimal function 
(see Fig. 3) is 0.578» + 0.465Dyp . It is thus not a continuous function 
of ¢ but has jumps at ¢ = +7'/2; this is characteristic of all of our 
optimization problems except for the special case 6° = 1. 

A note on previous work in the direction of Theorem 2. The con- 
nection between the extremum problem for 8 = 1 and the largest 
eigenvalue of (2) was noted by Chalk” and Gurevich,’ both of whom 
found the appearance of the optimal function without analytic solution 


0.9 





T 


0.8 


0.7 

















Fig. 3 Plot of optimal f(t) for a? = 0.977, 8B? = 0.96, 77/: 
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of the integral equation; the latter also plotted the largest eigenvalue. 
The set of eigenfunctions was recognized in this context by Ville and 
Bouzitat,’ who also performed a lot of numerical work. Finally Fuchs’ 
has stated, without proof, a theorem equivalent to Theorem 2. He 
considers n-dimensional spaces and Fourier transforms, and two 
arbitrary subsets of finite measure in the time- and frequency-spaces 
respectively. His proof, however, which we have been privileged to see, 
is quite different, and is not directed towards the properties of ® and 
® which have been our chief concern. Our present method is capable 
of broad generalization; some thoughts in this direction are given in 
the next section. 


IV. EXTENSIONS OF THE THEORY 


It is quite natural for us to ask what the real essentials of the study 
up to this point have been, and under what circumstances results 
similar to Theorems 1 and 2 could be obtained. Such an investigation 
will be reported in a separate paper;’ we should, however, note what 
some of the results are. For the relevant language, we refer the reader 
to Ref. 6. 

We have a Hilbert space £’, and two subspaces ® and D. The key 
property we require is that ® and D form a nonzero minimum angle; 
the latter property is equivalent to requiring that 


| BDBf || 
a 
j L? f 


sup 


It now follows that ® + D is closed, and we can again study the region 
of possible values of | Bf || and || Df || if || f |) = 1. We do not, however, 
obtain eigenfunctions analogous to {y,} unless the operator BDB is 
completely continuous. If, for example, £° is the space of square- 
integrable functions with respect to Lebesgue measure over n-dimen- 
sional Euclidean space R", if D is the subspace of functions vanishing 
outside of a bounded subset of R” of positive measure, and if @ is the 
subspace of functions whose Fourier transforms vanish outside of an- 
other bounded subset of R" of positive measure, then BDB is com- 
pletely continuous, and the full theory applies. 

As an example of a theorem which is again true in the general situa- 
tion, but is of interest also for timelimited and bandlimited functions, 
let us prove 

Theorem 3: Let an arbitrary function d © 9, and another function 
b © @®, be given. Then there exists an infinite collection S of functions 
f © &’ such that if f © S, then Df = d and Bf = b. There is a unique 





PROLATE SPHEROIDAL WAVE FUNCTIONS II 81 


fo © S of least energy, and there is a unique f; © SM (@® + D); fur- 
thermore, fo = fi. 
Proof: et us consider the function 


f* = > (1 — B)(DB)"d + DS (1 — D)(BD)". (14) 
0 0 


The first sum, for example, means 
d — Bd + DBd — BDBd + DBDBd — BDBDBd + ---. 
Since, for any g, || DBg || < VW o || g || and || BDg || S Wyo || g 
we know that the two series defined on the right side of (14) con- 
verge in: norm, with their sum defined as the function f* £. 
l‘urthermore, since f* is defined as a limit of functions in ® + 9, it is, 
by Lemma 2, itself in ® + D. So we may write 
ft = d* + dt, 
where d* € D and b* € @. 
Let us next compute Df* and Bf*. We have 


Df* = & (1 — DB)(DB)"d + > (D — D)(BD)"b; 
0 0 


all of the second series, and all but the first half of the first term of the 
first series, vanish. Hence Df* = d, and similarly Bf* = b. We have 
thus shown that f* ¢ SAM (@® + D); we can complete the proof that 
f* = f, if we can show that SNM (@ + D) contains no other function. 
Suppose thatf;=d,;+b;, i = 1,2 are bothin SN (® + D). Then 

d = Df, = dy + Dhi = dy + Dbe = Dfe (15) 
and 

b = Bf, = Ba, + by = Bd, te be = Bfe 
so that 

DBd, + Db, = DBd. + Dho. 

Hence, by subtracting (16) from (15), we have 


(1 = DB)d, = (1 — DB)d., 


(d, — dz) = DB(d, — dz). 


Since, however, || DBg || < V/Xo ||g || for any g, we must have 
d, — d, = 0, so that d; = d2. Similarly, b; = be, so that fi = fe, and 
thus f* is the unique member of SM (@ + 9). 

Now suppose x is any other member of S. We may write 
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r=f* +49, 
and since Dx = Df* d and Bx = Bf* b, it follows that 
is th 
But 


f* = d* + b* while LS +--S. 


Hence 


r 


, 


with equality if and only if g vanishes. Thus f* is also the unique mem- 
ber of S of minimum norm. An infinite number of other members of SS 
may be formed by adding to f* any of the functions orthogonal to 
® + D whose existence is guaranteed by Lemma 3. 


Note: Ifd = >> a,Dy; and b > bi; , then 
inh 


: a, ; b, — a;; 
ing mine acer y, 


so that, in particular, 


f* || $ FS (la + [1b )). 


APPLICATIONS 


5.1 Filter Theory 


Suppose we wish a filter to have an impulse response f(t) which 
vanishes for ¢ > 7. Such a filter clearly cannot be strictly bandpass; 
but how would we select the filter so that as much of the impulse re- 
sponse as possible is contained in | w < @ for some given 2? Suppose, 
by this, we mean to choose f(t) so that 


[1 Fle) f de 


7 


a 


| | F(w) | dw 
' ' 
= <0 
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is as large as possible, where 


" 
F(a) - | f(t) eo dt 


is the Fourier transform of f(t). Then the best choice is 


f(t) = wo(e+ 5 .e), 


where ¢ = Q7'/2, and y is the prolate spheroidal function of the 
present and the preceding papers. 
If, instead of requiring f(t) to vanish outside of (0,7), we ask that 


both 
/ | F(a) | dw = 8° 
J>2 


wW 


| + | | f(t) |? dt =a 
e) T 


be small while the total energy of the impulse response is fixed at unity, 
then Theorem 2 above gives the complete region of possible (a,8) 
values. 


5.2 Data Transmission 


When we choose a combination of pulse shape and transmission 
characteristic for a broadband data transmission system, we are in- 
terested in minimizing both the tail of a pulse outside its time slot and 
its spectrum outside of an assigned frequency band. Once again, it is 
not possible to make both of these ‘‘spillovers’”’ in time and frequency 
arbitrarily small; the above theory gives some information on inter- 
channel and intersymbol interference. For a theory which is more 
nearly complete, however, the relation between timelimiting and pass- 
bandlimiting (i.e., toQ; S |w| S Q) needs to be better understood ; 
while our general results apply, the identity of the optimal function 
Yo is not known in the case that B is projection of the transform into 
such a passband. 


5.3 Antenna Theory 


Let us consider a horizontal (s,¢) plane from which the strip | | < a 
of width 2a, to be called the aperture, has been removed. If the illumi- 
nation across the aperture is independent of s, then the amplitude of 
the field across the aperture may be represented by a function f(t) of 
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one variable, where |¢| < a. If we consider the resultant pattern of 
radiation in a distant parallel horizontal plane, then the field at a large 
distance from the aperture is proportional to 


| f(t) e' dt = F(u), 


a 
where u = k sin 6, k = 22/X, @ is an angle measured from the vertical 
through the center of the aperture, and \ is the wavelength. The Q of 
the antenna is then defined (equivalent to the definition of Woodward 
and Lawson;’ it is given explicitly by Kovacs and Solym4n’) as 


/ | F(u) |’ du 
= = ke a -. 


. 9 
| | F(u) | du 
-k 


Q 


* 


This may be rewritten as 


a 
I | f(x) |° dx 
a 


Q = — ne fi 
| | Bf(x) f° de 


as 


where B means limiting the Fourier transform of f to |u|! < k. Thus 
by the previous theory, 


l 
Qz--1, 
f=), 


where Ay = Ao(ak/2) is the first eigenvalue of (2) as defined in this 
and the preceding paper. We thus have an absolute lower bound on the 
() which can be obtained for given a and k. 
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Considerations on the Solar Cell 


By D. A. KLEINMAN 


(Manuscript received September 1, 1960) 


The collection efficiency in solar cells is treated by a new method in which 
all the effects of the solar spectrum and the absorption curve are contained 
in a single function readily obtained by numerical integration. The method 
is illustrated by a detailed study of the effects of surface recombination, body 
recombination and junction depth in silicon cells. The method is also gener- 
alized to include built-in electric fields, and calculations are given for sili- 
con. Sufficiently strong fields to improve the collection efficiency markedly 
can be produced in some compound semiconductors from a gradient in the 
energy gap. A discussion is given of the dependence of the collection effi- 
ciency on the absorption curve of the semiconductor. It is shown that silicon 
has a very favorable absorption curve in comparison with GaAs or InP. 
Finally, a treatment is given of the minority carrier collection in a two-junc- 
tion cell, and calculations are presented for silicon. It is concluded that this 
structure may be important for cells with high energy gaps and short life- 
limes. 


I. INTRODUCTION 


The considerations reported in this paper have been stimulated by the 
current interest in the solar battery as a power supply for instruments 
and transmitters in satellite and space probe vehicles. A number of 
space vehiclest have contained solar batteries with peak outputs rang- 
ing from several watts to several hundred watts. It has been demon- 
strated that solar battery power supplies are technically feasible, not 
only in space vehicles but also in terrestrial telephone systems.” For the 
latter type of application, however, the solar battery has been found to 
be not competitive economically with several other available power sup- 
plies.” However, it is highly advantageous for space vehicles because of 
its advanced development and commercial availability, light weight, 
reliability and long life. The long life is due not only to the ruggedness 
and permanence of its structure and the absence of moving parts or 

t For a review of the use of solar batteries in space see Daniels.' 
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chemically unstable components, but also to its external and independent 
source of energy, the sun. The belts of particle radiation® recently dis- 
covered surrounding the earth may materially reduce the life of solar 
batteries carried into space.’ We shall proceed with our discussion, 
however, on the assumption that the solar battery can survive the radia- 
tion. 

The solar battery is an array of hundreds or thousands of individual 
cells called solar cells. Each cell is a semiconductor slab, typically 1 X 2 
em, containing a p-n junction within a very small distance, typically 
2 X 10 ‘em, of the illuminated surface. The front and back surfaces are 
fitted with ‘‘ohmic”’ contacts for making electrical connection, and the 
front surface may be specially treated to reduce its reflectivity.” Com- 
mercially available cells are made of silicon, and have efficiencies of up 
to 14 per cent for converting the solar radiation incident upon them into 
electric power.’ Other methods now known for converting solar radiation 
are far less efficient.t Thermoelectric converters, for example, on which 
considerable work has been done, can approach efficiencies of 1 per cent. 

The solar cell was invented by Chapin, Fuller, and Pearson,” who 
briefly described its fabrication, its principles of operation, and the limi- 
tations on its efficiency. Due to the work of the inventors and later au- 
thors, notably Prince,’ Pfann and Van Roosbroeck,” Cumerow,” Ritt- 
ner and Loferski,” the solar battery is well understood in terms of 
concepts familiar in electrical circuits and semiconductor physics. 

High efficiency in a solar battery would be desirable in any application, 
but especially so in space vehicles. The value of the vehicle, launched at 
considerable expense, depends in large measure on the instruments and 
transmitters it carries and the power available for this equipment. At 
the same time, the more equipment that is carried, the less space and 
weight can be allowed for the power supply. Therefore, in the economy 
of space vehicles the solar battery should have the maximum possible 
efficiency irrespective of the costliness of the improvements. 

The operation of the solar cell and the losses of efficiency can best be 
described in five steps: 

(a) Radiation is incident upon the surface and some is reflected with- 
out entering the cell. This reflection can be a very important loss, since 


the reflectivity’’ of clean silicon is about 30 per cent in the wavelength 
range of interest (0.4 to 1 uw), and other materials that might be used 
also have high reflectivities. In practice it is found that the processing 


in the manufacture of silicon cells leaves the surface with quite a low 
ee § 16 . . 
reflectivity.” A recent study'” has shown that the best antireflection 


t For a review of the utilization of solar energy, see Ref. 8, especially Vol. V. 
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treatments can increase the short circuit current of silicon solar cells by 
20 to 25 per cent compared with clean surface cells. It is reasonable to 
expect that the reflection loss from any material that might be used in 
the solar cell could be minimized in the same way. 

(b) The light enters the cell and some is absorbed by the intrinsic 
absorption process in which a hole-electron pair is created and a photon 
is destroyed.t The light absorbed in this way is the useful light in the 
solar cell. Light of wavelength longer than the intrinsic absorption edge 
‘-annot produce hole-electron pairs and is wasted in the solar cell. There 
is a further waste of energy when hole-electron pairs are produced by 
photons with more than the minimum required energy, since the excess 
energy is transferred in a very short time to the semiconductor lattice 
in the form of heat. These losses may be ascribed to the spectrum of 
solar radiation, since they would not occur if the radiation were mono- 
chromatic at the wavelength of the absorption edge. For silicon the 
losses due to the solar spectrum are about 53 per cent of the energy which 
enters the cell.” By choosing a semiconductor with a somewhat higher 


1 . 13,14,15 py 
energy gapf this loss can be considerably reduced. Che energy gap 


is therefore very important in considering materials for the solar cell.§ 
(c) Some of the minority carriers produced by the light flow by dif- 
fusion to the p-n junction. These are the carriers which contribute to the 
output current of the cell. Other carriers diffuse away from the junction 
and recombine at the surface or deep inside the cell. The percentage of 
minority carriers which contribute to the current is called the collection 
efficiency. In a typical commercial silicon solar cell of 9 per cent over-all 
efficiency the collection efficiency is about 60 per cent.’ Minority carrier 
recombination is, therefore, a serious loss of efficiency in the solar battery 


: . 6,10,18 ¢ +: 
at the present time. From the spectral response of silicon solar cells 


we can infer that the surface recombination velocity is very high, proba- 

bly greater than 10° em/second. From the analysis to be presented we 

can also infer that the body lifetime is about 10 microseconds, the dif- 

fusion length|) about 10 “em. In high-purity silicon the lifetime can be 
“Ws 20,21 : . . 

several milliseconds’ and with certain surface treatments the recom- 


t For a review of the optical properties of semiconductors, see Hrostowski.!7 

t For the purposes of this paper the energy gap and the intrinsic absorption 
edge are essentially the same thing, except that the former is expressed as an 
energy while the latter is the equivalent photon wavelength. 

§ We shall not consider in this paper composite cells such as those suggested 
by E. Jackson (Ref. 8, Vol. V), in which one attempts to reduce spectrum losses 
by stacking several thin cells of different energy gaps. Cells of this type appear 
to be somewhat impractical from the standpoints of mechanical construction and 
of providing a suitably matched electrical load. 

Recently a measured value of ~5 X 107% em for the diffusion length has been 
reported by Vavilov, Smirnov and Patskevitch.'® 
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bination velocity~ can be as low as 40 cm/second. The recombination, 
therefore, is due almost entirely to the degradation” of surface and body 
lifetime that occurs in the manufacturing process. In compound semi- 
conductors the recombination losses are likely to be considerably greater 
than in silicon. In gallium arsenide, for example, the lifetime seems at 
present to be of the order of a millimicrosecond.” In general, it appears 
difficult to obtain lifetimes greater than 0.01 or 0.1 microsecond in the 
compound semiconductors. t 

(d) The diffusion maintains an excess concentration of minority car- 
riers on both sides of the junction. The voltage developed by the solar 
cell is due to these excess concentrations of minority carriers. This volt- 
age, however, is considerably less than the energy (in units of electron- 
volts) of a hole-electron pair in the semiconductor. The latter, for our 
purposes, may be taken to be the energy gap, which in silicon is 1.2 
volts.”* The voltage of a silicon solar cell in full sunlight under maximum 
power conditions is about 0.4 volts."”” Therefore, the cell is able to con- 
vert only a portion of the energy stored as hole-electron pairs into elec- 
trical work. The loss may be referred to as the junction loss. The junction 
loss should vanish and the voltage should approach the energy gap when 
the minority carrier density approaches the majority carrier density, a 
limit corresponding to infinite light intensity. In the other limit of zero 
light intensity, the junction loss causes the efficiency of the solar cell to 
approach zero. According to the equivalent circuit point of view of Pfann 
and Van Roosbroeck,” the short-circuit current of the cell flows partly 
through the load and partly through the junction in the forward direc- 
tion. The voltage and the junction loss therefore depend upon the for- 
ward current-voltage characteristic of the junction. The theory of p-n 
junctions’ predicts that the forward current should decrease exponen- 
tially with increasing energy gap. Therefore, insofar as actual junctions 
obey the ideal junction theory, the junction loss can be reduced by in- 
creasing the energy gap. A number of authors“ have considered the 
spectrum loss and junction loss as a function of energy gap. If all other 
losses are neglected, the maximum efficiency is obtained for an energy 
gap of about 1.6 volts.” It is now possible with mixed semiconductors” 
to obtain nearly any desired energy gap from 0.7 volt (germanium) to 
2.4 volts (GaP), which completely covers the range of interest. It should 
be kept in mind that silicon p-n junctions show a large contribution to 
the current from thermal generation and recombination through traps 


° . p . 29 ry “eas ; . 
in the junction region.” The short lifetimes in the compound semicon- 


ductors suggest that trap effects may be even more important in those 
t Ref. : 


. 24, p. 58 
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materials. Therefore, it may not be possible to obtain large reductions 
in the junction loss by increasing the energy gap. 

(e) Finally there is the loss due to resistance of the very thin side of 
the junction next to the surface and of the contact to the surface. This 
resistance places a lower limit on the depth of the junction. Prince” has 
considered the optimum depth for silicon solar cells taking into account 
the resistance loss and the collection loss. In practice, the internal re- 
sistance of silicon cells" is between one and two ohms and the junction 
depth is between 1 X 10 * and 2 X 10‘ em. The elimination of resist- 
ance loss would increase the efficiency of a 9 per cent cell to about 11 or 
12 per cent.”* Unlike the other losses considered here, the resistance loss 
is not characteristic of the material used in the solar cell, and the methods 
for reducing it will be similar for any material. 


It has been mentioned that the solar spectrum loss and the junction 
loss have been considered at some length. It is also well understood what 
to do about reflection and resistance. But the collection loss has not been 
thoroughly treated in the literature. Several calculations’ have been 
made using the approach of Pfann and Van Roosbroeck.” This may be 
called the monochromatic method. Light of a certain wavelength, and 
therefore having a certain absorption coefficient in the material, enters 
the solar cell. Solutions are obtained in terms of elementary functions 
for the minority carrier density, taking into account diffusion, surface 
and body recombination, the generation of carriers by the light and the 
boundary conditions at the junction and at infinite depth. The solution 
gives a certain diffusion current into the junction which is the short- 
circuit current of the cell. This current must then be averaged over the 
wavelengths in the solar spectrum to obtain the collection efficiency. The 
averaging requires a tedious numerical integration, because the mono- 
chromatic current is a complicated function of the absorption coefficient, 
which is in turn a rapidly varying (measured) function of wavelength. 
The method is sound and can account in a satisfactory way for the col- 
lection efficiencies observed in solar cells.° The method is not well suited, 
however, for a systematic discussion of collection efficiency and no such 
discussion has been given. Several authors’ have even argued that 
the collection efficiency can be considered unity in fundamental con- 
siderations on the solar battery, since lifetimes can be expected to in- 
crease as technology improves. A review of the history of the photovoltaic 
effect and its utilization has recently been given by Rappaport.” 

In this paper we consider the collection efficiency more systematically 
by a more powerful method than the monochromatic method. This 
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method is based upon obtaining a function, called the photodensity func- 
tion, which includes all the effects of the solar spectrum and the wave- 
length dependent absorption coefficient. This function is obtained by a 
relatively easy numerical integration over the solar spectrum. In terms 
of the photodensity function the solution can be obtained almost im- 
mediately to any solar cell collection problem in the approximation in 
which a single diffusion length describes the minority carriers. Illustra- 
tive calculations are presented for silicon solar cells showing how the 
collection efficiency depends on surface recombination velocity, junc- 
tion depth and diffusion length. The method is readily generalized to in- 
clude cases in which different diffusion lengths must be used for electrons 
and holes. Another generalization is presented which takes into account 
a “built-in” electric field in the region between the junction and the 
surface. It is shown that by the use of mixed semiconductors it should be 
possible to obtain sufficiently large built-in fields to increase the collec- 
tion efficiency significantly. In ordinary silicon cells, however, one would 
not expect the field to be large enough to have much effect. The im- 
portant question of which semiconductors should be best for solar bat- 


3,14,15 ,30 


° ° . 11,1 
tery applications, already much discussed 


with respect to spec- 
trum and junction losses, is taken up again from the point of view of the 
collection efficiency. It is pointed out that the absorption coefficient as a 
function of wavelength is very important in determining the collection 
efficiency. Silicon has an absorption curve of favorable shape, which in 
part accounts for its present superiority over other materials of more 
favorable energy gap. It is possible that for room-temperature use silicon 
will remain the best material, although higher-gap materials will cer- 
tainly be needed for use at temperatures above 200°C. Finally, there is 
presented a discussion of cells containing two junctions to improve the 
collection efficiency. It is shown that in cells of comparatively low col- 
lection efficiency (<50 per cent), considerable improvement can be ob- 
tained by the use of a second junction. This construction may prove im- 
portant in high gap cells which might otherwise have rather low 
collection efficiencies. 


Il. FORMULATION OF THE PROBLEM 


If the concentration of minority carriers is small compared to the con- 
centration of majority carriers, the equation describing the production, 
diffusion, and recombination of minority carriers is 


pAG ‘ 


i i : ; 
D' re. + N(A)a(vAye ™ dd = 0. 


dx? T Jo 
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In (1), n() is the excess minority carrier concentration over the equi- 
librium concentration. In a typical case the maximum value of n(z) is 
of the order 10” em *, so that the validity of (1) is assured. The diffu- 
sion constant D and lifetime 7 will be assumed to apply to all minority 
varriers whether holes on the n-side or electrons on the p-side of the 
junction. This assumption should lead to no serious error in silicon, but 
might have to be modified for some of the III-V semiconductors because 
of the relatively low mobility of the holes.t The integral term in (1) 
represents the production of minority carriers by light with a photon 
distribution N(A) in a material with absorption coefficient a(A) and 
intrinsic absorption edge \,. The total flux of photons capable of pro- 
ducing hole-electron pairs is 
rg 
N=| N(Q)a (2) 
“0 

We shall neglect reflection completely and identify N(A) with the solar 
photon spectrum with respect to wavelength.{ The total effective photon 
flux isN = 3.3 X 10’ em” sec ‘ for silicon.§ The boundary conditions 
to be imposed on n(x) are 


n(o) = 0, n'(0) = (s/D)n(0), (3) 


where s is the recombination velocity of the surface. A solution of (1) 
and (3) over the whole domain 0 < x < © represents the minority 
carrier density in a homogeneous illuminated semiconductor. 

If we now locate a junction at depth x = a, additional boundary con- 
ditions must be satisfied at the junction. In general, these will relate the 
minority carrier densities on each side of the junction to the operating 
voltage of the cell. The simplest case is the short-circuit condition, in 
which the voltage is zero.|| For this case the excess carrier densities must 
vanish on each side of the junction.”°”’ The boundary conditions for the 
short-circuit condition are therefore 


nia) = 0. (4) 


t Ref. 24, p. 12. 

t Details on the solar spectrum are given by Ref. 31. See also Ref. 15. 

§ This number, obtained from Ref. 31, is in substantial agreement with the 
plot of 4 N(A)dd in Ref. 15. We are considering the solar radiation in space just 
outside the earth’s atmosphere. 

| We neglect voltage drops due to internal resistance. This causes no loss of 
generality, since an arbitrary resistance can be included in the equivalent cireuit. 
In the presence of resistance, (4) corresponds to a small forward bias externally 
applied to the cell. The statement following (4) remains valid, and one identifies 
this maximum diffusion current, not the short-circuit current, with the current 
generator. 





92 THE BELL SYSTEM TECHNICAL JOURNAL, JANUARY 1961 


This corresponds to the maximum current which can be drawn from the 
cell; smaller currents correspond to nonvanishing values of n(a) on both 
sides of the junction. However, we may regard any solution of (1) cor- 
responding to arbitrary operating conditions as the superposition of the 
short-circuit solution and an appropriate solution of the homogeneous 
equation 


(5) 


According to p-n junction theory,”*”’ these solutions are just those asso- 
ciated with forward currents in an unilluminated p-n junction. The super- 
position of solutions for the minority carrier density is therefore equiva- 
lent to superposing the short-circuit current with an appropriate for- 
ward current in the junction. If J, is the short-circuit current and J; the 
forward current, the current in the external circuit is 7, — J;, in ac- 
cordance with the equivalent circuit of Pfann and Van Roosbroeck.” 

The collection problem, therefore, is to calculate the short-circuit solu- 
tion from (1) and (4) and from this the diffusion current density at the 
junction, taking into account the contributions from each side. The col- 
lection efficiency then is given by 


Q, = DN “(| n’(a—) | + | n’(a+)] ), (6) 


with the subscript “1” indicating that the expression applies to cells 
with a single junction. In a many junction cell Q, would be a sum of 
terms of the form (6). 

The equivalent circuit for a solar cell with load therefore consists of a 
current generator 7, connected to a junction and a load in parallel, with 
the currents J, flowing in the junction (forward) and J; — J, in the 
load. Pfann and Van Roosbroeck” obtained the following condition for 
maximum power delivered to the load: 


G=zlnz2+2z-1, (7) 


where 
eV/kT 
€ ? 
V is the voltage, and G is a dimensionless reduced current proportional 
, . ‘ ‘ ° ,° 96,97 
to J, . It is assumed that J; obeys an ideal junction characteristic 


I, = eAJo(e""*” — 1), (9) 
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where A is the junction area and Jo is a characteristic particle current 
density. The reduced current G is then defined by the relation 


G a es 'eAdo = (N, Jo)Q. ( 10) 
The over-all efficiency can be written” 


kT . (iInz)? 
omen tC OP stet 
W é G ’ 


where z is the solution of (7), and 


W =N” | N(A)(he/d) dd. (12) 


“0 


Clearly W is the radiant energy asborbed on the average to produce one 
hole-electron pair. For large G, (7) becomes 


G>1, (13) 
and (11) becomes 
(eV/W)Q. (14) 


Here eV is the work done on the load and W/Q, is the energy absorbed 
/ g) 
per carrier flowing in the load. The approximation of large @ is fully 


justified, since in full sunlight @ will be of the order 10’. The efficiency 
for large G can also be written 


e, ~ (kT/W)Q, ln G, (15) 


which exhibits the logarithmic dependence of the efficiency on the light 
intensity. It will be noted that the over-all efficiency «, depends upon 
the collection efficiency Q, through the factor Q, and also through G. 


Ill. THE PHOTODENSITY METHOD 


The collection problem consists in solving (1) subject to (3) and (4). 
We begin by observing that the function 


AG 
/ N(A)r atte _¢ “dh, (16) 
0 = oa 


where L = (Dr)! is the diffusion length, is a particular solution of (1). 
It satisfies the boundary condition at infinity but none of the other 


boundary conditions. If ambiguity arises due to the pole al = 1 oc- 
curring in the range of integration, the integral can be taken as a prin- 
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cipal value. The pole is removed if we combine (16) with an appropriate 
solution of the homogeneous equation (5), and write 


“AG 


P a(Xd) 
NX)» > N(A)r 


ax r/ L ~ 
—— (¢ — ¢ ) dX. (17) 
v0 | = a? l? : 


This function is a solution of (1) and satisfies 


n(co) = n(0) = 0, (18) 


which is the form taken by (3) when the recombination velocity s is 
very large. Therefore, n(x), represents the density of excess minority 
carriers in a semiconductor without junctions illuminated on a surface 
with fast recombination. The derivative of n(2), at the surface is 


n'(0), = a % (19) 


We call n(x), the photodensity. It can be readily evaluated by numerical 
integration (see Appendix), since the integrand is a simple expression 
with no poles. 

It is convenient now to introduce dimensionless quantities 


f= 2/L, 8 = al, yy nD. 
v(d) = N(A)/N, = Did. 


and define the function 


Then the photodensity can be written 


N( 2X) p (Nr/L)F(¢,L), 


where F(¢,L) is the photodensity function 


Ac 


P(t) = tet [7 a) 8 


= , lF — B)| dx. (22) 


This function is shown in Fig. 1 as a continuous function of ¢ for three 
. 4 

values of L (10 

absorption curve for silicon.” + The calculation of F (¢,L) is discussed 


, 10°, 10 ~ em) based on the solar spectrum and the 


in the Appendix. For small ¢ the photodensity function has the ex- 


pansion 
A OY as. B(d) al oe 
FUL) =¢ | v(r) ; dx — 3° | vV(A)B(A) dA + 


“0 A “0 


¢ Values of a > 10° cm™! required as discussed in the Appendix for the cal 
culation of Fig. 5 are from Pfestorf.*8 
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Fig. 1 — The photodensity function (22) for silicon as a function of ¢ for L = 
Ww, 40- “3 and 10-2 em. 


Once F(¢,L) has been obtained, the solution n(x) satisfying (1), (3) 
and (4) is readily constructed from F'(¢,L) and the functions e*', which 
satisfy the homogeneous equation (5). The solution may be written 

n(x) = (Nr/L)[Aé + Be! + F(¢,L)], (24) 


where for 0 S 


1 F(A)(1 + w) + we "F'(0) 

2 sinh y + w cosh y 

yl — w) + we’F’ (0) 
“eet w cosh ¥ 


and fora S 
A 
B —e'F(y), 


(26) 


where F’ = dF /dé and for brevity L has been omitted as an argument 


of F. The solution in the region a S x < = is, of course, independent of 
In the region 0 S x S a the solutions in the limiting cases w — 0 
and w— * are 


n(x) - 


n(x) - mh f F(y) |, (27) 
' sinh ¥ 


: sinh (y — ¢) 
F(¢) — F(y) + ——1—* F'(0) |. (28) 
cosh ¥ 
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The boundary conditions satisfied by these solutions at x = 0 are 
n(0) = Oand n’(0) = O respectively. 
The collection efficiency for single junction cells can now be obtained 
from (6) in the following form: 
F(y)(1 + w) e” + wF’(0) 


Q « — ee eee. (29 
a sinh y + w cosh y ) 


In the limits w — 0 and w > - this reduces to 


Q— F(y)(1 + cothy), 


w->() 


Q: — F(y)(1 + tanh y) + F’(0) sech y, 
wre 
which refer to fast and slow surface recombination respectively. 

Much of the simplicity of (29) and (30) results from the assumption 
of ‘effective’ values for D and + which apply to both sides of the junc- 
tion. The difficulty of measuring these parameters in an actual cell, es- 
pecially as a function of depth, justifies this simplification for most con- 
siderations. The most conspicuous case where this assumption may lead 
to serious error is in some of the compound semiconductors where the 
mobility and diffusion constant may be an order of magnitude less for 
holes than for electrons.t When different L obtain on the two sides of the 
junction (24), (25) and (26) are still formally valid but (29) is no longer 
valid. The collection efficiency can then be written Q; = Q. + Q,, 
where 

_ F(y)(cosh y + w sinh y) + 


Q_ = Bi I Bi 
sinh y + w cosh y 


Q, = Fly) + F'(y) 


are the contributions from 0 S « S aanda S x < ©& respectively. If 
the same L is used for both sides of the junction Q_ + Q, reduces to 
(29). 

The fast surface recombination limit w = 0 is valid if w satisfies the 
two conditions w < 1, w < y. In sillicon solar cells the junction depth is 
considerably less than the diffusion length (y « 1) so that the second 
condition implies the first. Therefore the criterion for high surface re- 
combination velocity is 


s > D/a. (32) 


The diffusion constant D can be obtained from measured mobilities u by 


t Ref. 24, p. 12 
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° - P . 34 ° . . ‘ 
means of the Einstein relation,” which in conventional laboratory units 
is 


D (in em’/second) = 0.026 (sre) uw (in em’/volt-second), (33) 
300°K 

with 7 the absolute temperature in °K. The mobility in highly doped 
silicon has been studied by Backenstoss,” who finds that for impurity 
concentrations greater than 10 cm * the mobilities of electrons and 
holes approach the limiting values of 80 and 40 cm’/volt-second respec- 
tively. Since a ~ 10‘ cm, the criterion (32) gives s > 10° em/second in 
silicon. Although surface treatments are known” which can reduce s in 
pure silicon to below 10° cm/second, Prince and Wolf® point out that 
these treatments tend to increase significantly the reflectivity of the 
surface. If this occurred the treatment might well reduce rather than 
increase the over-all efficiency of the cell. On the other hand, Malovet- 
skaya, et al.’° have reported antireflection films that greatly reduce re- 
flection losses without increasing the surface recombination. The effect 
of surface recombination on Q, is shown in Fig. 2 for a silicon solar cell 
having the typical parameters 


D = 5 em’/second, 


T 2 X 10 ° second, 


L 10°” em, 


10°* em. 


10 107 
S IN CM/SEC 


Fig. 2 — The collection efficiency as a function of surface recombination ve 
locity for a silicon cell having the parameters (34). 
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For this cell Q; increases from 63 to 87 per cent as s decreases from 10’ 
to 10° em/second. This example shows that considerable improvement in 
Q; would result from the elimination (s < 10° em/second) of, surface 
recombination. The presence of surface recombination in commercial 
silicon cells is indicated by the rapid fall-off in the spectral response’’”* 
at short wavelength \ < 6 yw. Little is known about surface recombina- 
tion in the compound semiconductors. The fact that body lifetimes tend 
to be short™ (r ~ 10° second) suggests that surface recombination is 
probably very fast (s ~ 10‘ em/second ). On the basis of these considera- 
tions, the most reliable guess is to set w = 0 except where specific infor- 
mation to the contrary is available. 

The dependence of Q; upon the junction depth is shown in Fig. 3 for 
a silicon cell with D, 7 and L as given in (34). The curves shown for 
w = 0 and w — & are envelopes for the entire family Q,(y,w). Although 
the limiting curves have quite different forms they approach the same 
form for small and large depths: 


‘AG 
a—- F’(0) = | v(A)B(A)(1 + BY" AD, 

aa ? (35) 
1 + 2F ( ¥Y - 


yYos 


The monotonic decrease of Q;(y,0) with increasing y is typical of silicon 
7 £Y YI 


cells. In semiconductors of considerably higher energy gap the condition 
F’(0) + 3F7(0) > O may be satisfied, in which case Q;(y,0) will at first 


























10-1 


_— 


Fig. 3 — The collection efficiency as a function of junction depth for a silicon 
cell with parameters D,7,L given by (34). The abscissa is y = a/L, where a is the 
junction depth and L the diffusion length. The curve Q:(y,~) refers to zero and 
Q,(y,0) to infinite surface recombination velocity 
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increase with y to reach a maximum and then fall to zero as y is increased 
further. A very broad maximum in Q;(y, ) is seen in Fig. 3 near y = 
0.2. The shape of the curve shows that relatively deep junctions (y ~ 
0.5) could be used if surface recombination were absent, which would 
permit the internal resistance of the cell to be greatly reduced. On the 
other hand, deep junctions cannot be used if the surface recombination 
is important. This calculation also shows that in a cell with surface re- 
combination reducing a from 10 * (y = 0.01) to5 X 10 *em (y = 0.005) 
would improve the collection efficiency only slightly. Therefore, one may 
regard a = 10 * cm as close to the optimum junction depth for silicon 
solar cells. 

The dependence of Q; on diffusion length is shown in Fig. 4 for a sili- 
con cell with a junction depth a = 10 * em. The curves Q,(L,% ) and 
(),(L,0) are envelopes for the family Q,(1,w) considered as continuous 
functions of L. These calculations show that L > 0.1 em may be con- 
sidered essentially infinite. The limiting values as L — ~ are 

Qi(L,*2) — 1, 
L 


x 


AG 


Q,(L,0) > »(X) 


1 — aa 


“0 ad 


dx, 


which are indicated in the figure. It will be observed that Q; increases 
relatively little as L increases beyond 10 ~ cm. On the other hand con- 
siderable improvement results from increasing L from 10 “ to 10° em. 




















a 6 1073 2 a 6 1072 
L IN CENTIMETERS 


Fig. 4 — The collection efficiency as a function of diffusion length for a silicon 
cell with the junction depth 10~* em. The curve Q:(L,~) refers to zero and Qi(L,0) 
to infinite surface recombination velocity 
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IV. THE EFFECT OF AN ELECTRIC FIELD 


In his review, Rappaport” has suggested that a “built-in” electric 
field may exist near the surface of solar cells causing minority carriers 
to drift toward the junction. If sufficiently strong this electric field would 
in effect eliminate surface recombination. One would expect that to re- 
duce surface recombination significantly the field would have to be of 
the order E ~ s/n ~ 10° volts/em, assuming the typical values s ~ 2 X 
10° em/second, np ~ 200 cm’/volt-second. The built-in electric field could 
arise in two ways, from a gradient in the impurity concentration, and 
from a gradient in the energy gap. In the first case the field is given by*® 


E ~ (0.026/a) In (No/Na)  volts/cem, (37) 


where Ny ,N, are the carrier concentrations at x = 0 and x = a respec- 
tively, and 0.026 = kT/e at room temperature. This is just the field 
required to cancel the diffusion current in equilibrium according to the 
Einstein relation (33). Actually (37) is an approximation giving an 
average effective field over the region 0 S x S a. Consider as an example 


a cell made by the diffusion of boron from the vapor phase into n-type 
6,10 


3 


silicon.°"*"" The surface concentration of boron is about'' Ny ~ 18" em 
and N, ~ 10" em *. If a~ 10 ‘em, the built-in field is E ~ 500 volts 
em, which is probably much less than s/u and therefore not large enough 
to cause much reduction in surface recombination. We may conclude 
that in commercial solar cells the built-in field can be neglected, as in 
the treatment of the last section and in the previous literature. On the 
other hand, the possibility remains that much larger built-in fields could 


be obtained, since Ny might be made to approach the solid solubility,”’ 


which for boron exceeds 10” em™*. The built-in field would then be 
about EK ~ 1800 volts/cm, which might cause a significant improvement 
in collection efficiency. 

It appears that really large fields EF > s/u might be obtained from 
gradients in the energy gap. Such a gradient could be obtained in GaAs 
by diffusing in phosphorous from the surface. Mixed crystals’ t GaAs- 
GaP can exist in all proportions, and the energy gap varies as a function 
of composition from 1.4 (GaAs) to 2.4 (GaP). If the impurity concen- 
tration is approximately constant, the band edge corresponding to the 
majority carriers must also be approximately constant. The gradient of 
the energy gap therefore appears almost entirely in the band edge for 
the minority carriers, and is equivalent to a built-in electric field acting 


t See also Ref. 24, p. 52. 
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on the minority carriers. It seems quite feasible that the field may be of 
order E ~ 10‘ volts/em in GaAs-P. In addition to producing the field, 
the gradient in energy gap causes the absorption coefficient to become a 
function of position a(A,z). To make the analysis more tractable we may 
regard a(\) as an “effective” absorption coefficient independent of posi- 
tion. 

To take into account a constant electric field it is only necessary to 
add to the left side of (1) the term —yE(dn/dz), giving 

dn on 


dn - ; 
De - =— = + i N(A)a(A)e dd = 0. (38) 
dx* ot dx 0 


This is conveniently written in the reduced form 


, dy 


d'y 
dt 


— y — 2 
de? d 


AG 
+ vBe ™ dx = 0, 
0 


where 
_ £- 


& = . — 
2D/nL’ (40) 


n 

4 Ne/L’ 

and 8,»,¢ are defined in (20). The boundary conditions on y(¢) are 
y'(0) = Sy(0), y(y) = 0, y(2x) = 0, (41) 

where 


S=2+0! (42) 


depends on the electric field as well as the surface recombination ve- 

locity. If the last term in (39) is dropped the resulting homogeneous 
ation has tl lution e ”. e”. whe 

equation has the solution e ”, e°, where 


p=(1+8)'-ses 
co=(1+8@)'+e2z 
po = 1. 
The general solution can be written 


y(¢) = Be ™ + Ae™ + F(¢,L,8), 
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where 


dr Bf t 
G B ¢ Sie e p 


F(¢,.Lg) = th 


S. neers 
“0 l + pB l — of 


For the region 0 S ¢ S vy, 


eye F'(0) + (S — a)Fly)e” 
pt ce—Pt?y + S(1 — e tr) 


F’'(0)e 9?” + (S + p)Flye” 


p + ce~’toy + S(1 — eter) * 
As & approaches zero, F(¢,L,&) approaches F({,L) defined by (22) and 
A,B of (46) go over into A,B of (25). It will be assumed that the elee- 
tric field exists only in the region 0 S ¢ S y. Thus the solution in the 


region ¢ = y is 
y(t) F(¢,L) — F(y,L)e"*. (47) 
Since y vanishes at the junction, the current is just the diffusion current 
and the collection efficiency is Q; = Q_ + Q@, , where 
Q+ = y'(v+), 
Q. = —y'(y_). 


From (47), 


QQ, Fiys,) + F’(y,) 
as in (31). From (44) and (46), 


F(y){1 er Gindhbihie Nilo +. pe itis + F'(O)« ad +- go) 


Q mae 
p + oe Fy + SL — e-@Fr) (50) 


— F’(7_), 


where F’(¢) = (d/df)F(¢,L,&). In the limit & — 0 this goes over into 
Q_ given in (31). If 28 > 8, 26 > 1/8 and 2& > y, (50) becomes 


AG 1 ekg 
Q,—- vil —e”’) dA — — | vBe ”™ dy. (51) 
oo 0) 2 “0 
Since the first term of (51) is the probability of absorbing a photon in 
the layer 0 S ¢ S y, the limit 6 — ~ corresponds to complete collec- 
tion of minority carriers produced in this layer. 
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A plot of Q— as a function of & is shown in Fig. 5 for a silicon cell 
with the parameters 
D = 5 em’/second, 


nu = 190 em*/volt-second, 


r = 2X 10” second, 


L 10° em, 
a = 10 ‘em, 
s = 2 X 10° em/second, 
2D/uL = 5.3 volts/em, 
s/n = 1050 volts/cm. 


The electric field is therefore E = 5.38 volts/em. The value of Q, cor- 
responding to these parameters is 0.52, so that the collection efficiency 
Q, = Q. + Q, varies from 0.67 at zero field to 0.87 at infinite field. It will 
be observed that these values correspond in Fig. 2 to s = 2 X 10° em/ 
second and s — 0 respectively, as one would expect. The critical field 
s/u indicated in the figure is the field at which about half the surface 


0.38 


























& 


Fig. 5 — The collection efficiency as a function of built-in electric field for a 
silicon cell with parameters (52). The abscissa is §& = Eyl /2D and 2D/pl = 5.3 
volts/em. The critical field s/u and the asymptotic limits of Qi are indicated by 
dotted lines. 
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recombination has been eliminated. Although it would be difficult to 
obtain built-in fields much larger than this in silicon, the high-field por- 
tion of the curve shows qualitatively the results to be expected from a 
gradient in the energy gap in a solar cell made of a compound semicon- 
ductor such as GaAs-P. The asymptotic formula (51) holds only for 
much higher fields than & = 10‘ because of the extremely high values” 
of a near \ ~ 0.3 yw. The asymptotic limits at low and high fields are 
indicated by dotted lines. For estimating the effects of a given field it is 
convenient to regard the field as effectively lowering the surface recom- 
bination velocity. For estimating the effective velocity s* one may use 
the recipe 


s* = s[{l + (Hy/s)(1 + sa/D)| “* (53) 


The collection efficiency can then be estimated from (29) by using an 
effective surface parameter w* 
than by evaluating (50) exactly. 


= D/Ls* with considerably less labor 


V. THE EFFECT OF THE ABSORPTION CURVE 


The absorption curve is the curve obtained by plotting the absorption 
coefficient a(A) as a function of wavelength \. Different semiconduetors 
have quite different absorption curves as well as different energy gaps. 
In the several studies that have been made on the effect of the energy 
gap no consideration has been given to the absorption curve>™'"°"*°"? 
All of these studies have been concerned with spectrum and junction 
losses rather than collection losses. From the standpoint of collection 
efficiency, however, the absorption curve is of essential importance if 
there is any considerable surface recombination in the cell. Although the 
elimination of surface recombination may be in prospect, it is still of 
interest to consider the effect of the absorption curve in a cell with es- 
sentially infinite surface recombination velocity. This provides another 
standpoint from which to discuss the advantages of different semicon- 
ductors. 

In Fig. 6 are shown the absorption curves for silicon,” GaAst and 
InP,” in the range 10° < a < 10° cm“. It will be observed that the curve 
for silicon differs from the other two in its gradual drop off with increas- 
ing wavelength. For a < 10° cm‘ it falls off more steeply, although not 
as steeply as GaAs or InP. The reciprocal a(X) ' of the absorption coeffi- 


+ The solid curve for GaAs is based on unpublished transmission measurements 
by W. G. Spitzer. The dashed part of the curve is an extrapolation based on a 
single additional reflection measurement by R. J. Archer giving a ~ (1 + 0.3) X 
10° cm™! at 0.546 yu. 
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Fig. 6 — The absorption curves for silicon, GaAs and InP. 
cient is a measure of the penetration distance of light of wavelength X. 

There is some wavelength A, , for which 
/ ‘ 1 ~ 
a(\,) ~ a, (54) 


which is the shortest wavelength of light useful in the cell. Light of 
shorter wavelength that \; is absorbed too close to the surface to be col- 
lected at the junction. Similarly, there is a wavelength 2, for which 


a(d\2)  ~ L, (55) 


which is the longest wavelength of useful light. If L = 10°° em or larger 
\» is nearly equal to the intrinsic absorption edge \¢ corresponding to 
the energy gap. Light of longer wavelength than \, is absorbed too far 
from the junction to be collected, or else is not absorbed at all. Since 
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the effective light lies in the range \; S A S 22 an effective photon flux 
‘an be defined: 


ed 


N(A) dy. (56) 


Nett = 


7 Ai 
The collection efficiency may be written 
Q, — Nett/N, (57) 


where the flux N is defined by (2). The limits A; , Ax, and hence Q, evi- 
dently depend in an essential way on the absorption curve. 

The first equation (30) gives Q, for the present case in terms of the 
photodensity function F(¢,L,) defined by (22). Therefore, the qualita- 
tive considerations given above enter into determining the value of 
F(y,L). According to (22), F(y,L) contains an integral over \ of the 
product of two functions 6B/(1 + 8) and ¢g[y(1 — 8)]. The weighting 
function v(A) representing the solar spectrum is slowly varying and not 
relevant to the present discussion. Fig. 7 shows plots of the two functions 
for a silicon cell with the parameters (34). It is clear from this figure 
that A; and Az correspond to cutoffs in g and 8/(1 + 8) respectively. The 
integrand in (22) is evidently small except in the region between the 
vertical lines in the figure corresponding to aa = 1 and al = 1. 

















Fig. 7 The functions 8/(1 + 8) and ¢ entering into the integrand of (22) 
defining the photodensity function. Calculation refers to a silicon cell with a = 
10-4, L = 107? em. Dashed lines show cutoffs \; , Ax corresponding to (54) and (55). 
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It is now clear that when ), lies very close to A, the collection efficiency 
will be relatively low. This is the case with very steep absorption curves 
like that of InP. On the other hand, a gradual absorption curve like that 
of silicon leads to relatively high collection efficiency. If \; , A» are de- 
fined by a(Ay) = 10° and a(A2) = 10° em‘, (57) predicts the collection 
efficiencies 0.70, 0.21 and 0.045 for silicon, GaAs and InP respectively. 
The value for GaAs is subject to considerable uncertainty because it is 
based on a little-known part of the absorption curve. In any case, all of 
these numbers are qualitative, and mainly of interest to show that. if 
surface recombination is high silicon is much superior to GaAs or InP. 
It is important to notice that this should be true in spite of the higher 
energy gap of the other two materials, because the differences in effi- 
ciency predicted for these materials on the basis of their energy gaps 
are much smaller than the differences in collection efficiency obtained 
here. For example, Loferski” finds that silicon and GaAs have relative 
efficiencies of 0.21 and 0.24 respectively, whereas silicon may have three 
times the collection efficiency of GaAs and 15 times that of InP. 

Several investigators have studied GaAs photocells. Gremmelmaier” 
obtained an over-all efficiency of about 4 per cent and a collection effi- 
diency of about 0.2, in agreement with the estimates given above. An 
over-all efficiency of 6.5 per cent for a cell of very small area (0.007 
cm’) has been reported by Jenny, Loferski and Rappaport.” These au- 
thors give current and voltage information for a somewhat larger cell 
(0.059 em’) having an over-all efficiency of 3.2 per cent, from which it 
can be deduced that the collection efficiency was about 0.26. It may be 
significant that the efficiency of 6.5 per cent was apparently obtained 
using a very low level of illumination (0.0057 watt/em’*). Also, very low 
collection efficiencies have been reported for InP cells by Rappaport,” 
although he attributes this to poor ohmic contacts to the cell. According 
to the present argument GaAs and InP will always give low collection 
efficiencies unless surface recombination can be eliminated, either by 
reducing the surface recombination velocity to below 10° em/second, or 
by means of a built-in electric field of order 10‘ volts/em. 

It should be mentioned that the spectral response for a GaAs cell re- 
ported by Jenny et al."' is not at all in agreement with the assumption 
of fast surface recombination. These authors observe an almost constant 
quantum efficiency from 5 to 9 yu. This differs radically from an earlier 
measurement by Seraphin,t which showed a pronounced fall-off at short 
wavelengths characteristic of surface recombination. No explanation has 
been advanced for this result nor can any be offered here. It may mean 


Tt Ref. 24, p. 65. 
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that surface recombination can be effectively eliminated in GaAs cells 
by treatments not yet defined, understood or controlled. 


VI. THE TWO-JUNCTION CELL 


In the typical silicon solar cell having the parameters (52), about 20 
per cent of the minority carriers produced are lost to surface recombina- 
tion and 13 per cent to body recombination. Therefore large reductions in 
surface recombination will be more rewarding in terms of over-all effi- 
ciency improvement than will proportionate reductions in body recom- 
bination. The two preceding secitons have been concerned primarily 
with surface recombination. It is also worthwhile to consider body re- 
combination in some detail, particularly since high-temperature applica- 
tions may be in prospect which will require high energy gap material.” 
As discussed in the introduction, body lifetimes tend to be short in these 
materials. From Fig. 4 one may estimate that Q, is likely to be no larger 
than 0.2 in a cell with short lifetime. How this low collection efficiency 
may be improved by means of a second junction is discussed in this sec- 
tion. 

The collection efficiency for a two-junction cell will be denoted Q, . 
If the second junction is located at depth b below the surface, and 
4 = b/L, the solution n(x) in the region a S x S bis 


L 


N , sinh(n — ¢) ,, sinh(¢ — y) ,, me 
nix) = : F(¢) oe " : F(y) = Sekt, F(n) , (38) 


sinh(» — y) sinh(m — y) 


which satisfies (1) and the boundary conditions n(a) = n(b) = 0. In 
the region x = b the solution is 


n(x) = = IF(S) — e" F(n)). (59) 


4 


The solution in the region 0 S x S a is the same as that obtained in 
Section III and given by (24) and (2 
combination w = 0, this is 


Nr sinh ¢ 
(xz) =- F(¢) — ~~ F(y) |, (60) 
ane L | , sinha" | 


satisfying n(0) = n(a) = 0. From (6), the total collection efficiency is 


5). For the case of fast surface re- 


Q. = F(y){[cothy + tanh $( — y)] + F(»)[1 + tanh $(m — y)J. (61) 
As 7 — 7, Q2 — Q, for the case w = 0 given by (30). The effectiveness 
of the second junction may be measured by the quantiy 


6 = (Q2 — Q)/Q1, 
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where Q, is the collection efficiency for a cell with a single junction at 
depth a. From (61), 


in F(n) 1 + tanh 3(m ~+) Lo io 1(n — y) 
F(y) 1 + coth y 1 + coth y 











Fig. 8 shows 6(n,y,L) plotted as a function of 7 for a silicon cell with 
a = 10 ‘cm and L = 10“, 10° and 10” em. These calculations show 
that 6(7) has a well-defined maximum which defines the optimum depth 
for the second junction for given a,L. 

Due to the loading effect of the second junction the relative improve- 
ment in over-all efficiency will be somewhat less than 6. This effect may 
be taken into account by use of the Pfann-Van Roosbroeck” efficiency 
expression (15). For the two-junction cell the reduced current may be 
written 

Gz = 3(Q2/Q)G1, (64) 
where G, = I[,/eAJo = (N/Jo)Q, is the reduced current for the single- 
junction cell. Thus the efficiency expression for the two-junction cell is 
, kT 


€&2 ~ 


2 Ww Q2 In Go . (65) 


The relative improvement in over-all efficiency can be measured by 


‘e=— €)/€ F (66) 





























Fig. 8 — The improvement of collection efficiency (defined by (62)) as a func- 
tion of the depth of the second junction for a silicon cell with first junction at 
depth 10-4 and ZL = 10~‘, 10-* and 10~? em. 
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where « given by (15) is the efficiency of the single-junction cell with 
the junction at depth a. Finally, the over-all improvement 6’ can be 
written as a function of the collection improvement 6: 

f mo (1 ef eee. (67) 

In G 

For practical purposes one may set In G = 18 in (67). The second term 
gives the effect of the loading due to the second junction. Two extreme 
cases may be noted: if 6 > 1, the second junction is a more effective 
collector than the first function and 6’ > 6; if 6 < In $/In G ~ 0.04, the 
second junction acts mainly as a load on the first junction and 6’ < 0. 
A summary of calculations for silicon two-junction cells is presented in 
Table I. In each case it is assumed that the first junction is at a = 10“ 
em and the second junction at the optimum depth given in the third 
column. The collection improvement 6 and over-all improvement 6’ are 
given in the fourth and fifth columns. In the second column is given the 
single junction collection efficiency in agreement with Q;( 1,0) of Fig. 4. 
The last row with L = 10° em is the most typical of good silicon solar 
cells. For this case the improvement in collection efficiency is only 
6 = 0.092. On the other hand, for L = 10 * em, 6 = 0.44 and 6’ = 0.42. 
This case applies qualitatively to any short-lifetime cell, and shows that 


the two-junction structure may be useful for improving the efficiency of 


high energy gap cells. 


VII. SUMMARY 


In this paper the present status of solar cell theory has been reviewed, 
with emphasis on clearly defining the various mechanisms causing losses 
of efficiency. This review leads to the conclusion that the problem of the 
collection of minority carriers has not received attention in the literature 
commensurate with its importance. 

The collection problem is formulated and then solved by a new method 
in which all the effects of the solar spectrum and absorption curve are 
contained in a single function, the photodensity function. The method is 
convenient for most solar cell collection problems and especially so when 
a single diffusion length can be used. 


TABLE I 


6 (in cm 5 (max 


0.186 0.032 0.443 0.417 
0.420 0.013 0.258 0. 226 
().634 0.008 0.092 0.055 
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The familiar single-junction silicon cell is considered first and caleula- 
tions are presented to show how the collection efficiency varies with sur- 
face recombination velocity, junction depth and diffusion length. It is 
found that the elimination of surface recombination would not only im- 
prove collection efficiency but also permit the internal resistance to be 
greatly reduced. Little or no improvement in silicon cells is to be expected 
from making the junction depth less than 10 * em or the diffusion length 
longer than 10” em. 

The theory is extended to include a constant built-in electric field, and 
calculations are presented for silicon. It is concluded that commercial 
silicon cells do not have large enough built-in fields to affect the collee- 
tion efficiency significantly. Sufficiently large fields should be obtainable 
in some compound semiconductors from gradients in the energy gap. An 
approximate relation is given for an effective surface recombination 
velocity less than the true velocity, which takes into account the effect 
of the built-in field in reducing surface recombination. 

A discussion is given of the dependence of the collection efficiency on 
the absorption curve of the semiconductor. This discussion provides an- 
other basis beside the energy gap on which to compare different semi- 
conductors for solar battery use. It is shown that silicon has a very favor- 
able absorption curve in comparison with GaAs and InP. It is suggested 
that this accounts to a large extent for the continuing superiority of 
silicon over other materials with larger energy gaps. The theoretical 
superiority of GaAs and several other higher gap materials over silicon 
can only be realized if surface recombination can be drastically reduced ; 
specifically, it is necessary that the effective surface recombination ve- 
locity be reduced to 10‘ em/second or lower. 

Finally, a two-junction cell is considered in connection with the prob- 
lem of reducing body recombination. Although body recombination is 
considerably less important than surface recombination in good silicon 
cells, it will be probably much more important in high energy gap cells. 
The illustrative calculations presented for silicon also have qualitative 
significance for other cells. It is shown that an improvement of about 42 
per cent in over-all efficiency may be expected from the two junction 
structure in material with a diffusion length of 10 * em. The improvement 
in a good silicon cell with a diffusion length of 10°* em would only be 6 
per cent. 
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™ i nl P = 
Note. Since the completion of this work a review paper by Wolf” has 
appeared which discusses some of the topics taken up here. 


APPENDIX 


In this section some comments are given which may be helpful for the 
evaluation of the photodensity function (22). A very accurate and con- 
b ° . " 44 ° ° ° 
venient method is the Gauss quadrature,” which approximates the in- 
tegral by a summation over the values of the integrand at certain specific 
(not equidistant) points. The limits 0,A¢ in (22) can be replaced by 
A, , Az so chosen that the contributions from the regions 0 < \ < A, and 
Ae < A < Agare negligible. The approximation is to represent the integral 


erg 
F(¢,L) = g(A) dr 


40 
HY 


~ | g(r) ar 


“Ay 
10 
= 3(r2 — A) DL Rig(A;). (68) 
j=1 
The choice of a 10-point Gauss quadrature here is arbitrary, but has 
proved to be a satisfactory compromise between convenience and ac- 
curacy. For silicon one can choose \; = 0.42 yw, A» = 1.08 yw, and (68) 
becomes 


[om dd = >> (0.33R;)g(d;). (69) 


The wavelengths \; and corresponding absorption coefficients” a(A;) and 
solar spectrum” weights v(;) for this case are given in Table II, along 
with the quadrature weights 0.33R; . The integrand g(\) can be readily 
evaluated with a slide rule for each X; . 


TABLE II] 
0.33R in cm™!) 


.0220 
.0492 
.0723 
.O888 
.0977 
.0977 
.0888 
.0723 
.0492 
.0220 


7X 104 
0(4) 
0(3) 
3(3) 
.2(3) 
03 (3) 
.5(2) 
.56(2) 


Cm w 


“Ind = & = b+ 


_— 
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The accuracy of (69) may be tested with an elementary integral hav- 
ing a qualitative behavior similar to g(A) in the range of integration. 
Such an integral is 


1.08 

/ (1.08 — r)*(r _- 0.42)! dr = 0.17106. 

0.42 

The quardature (69) with A; and 0.33R; from Table II gives 0.17111. 
The derivative F’(¢,L) is defined by the integral 


AG t(1—B) 
f/f " e B 1 — Be 
"(¢L) = [ ( aes. eS 
Pw) 0 oe He ne 1—8 me 


t @ ; B 
=e v(Xd) - {1 — Beele(1 — B)]} dd. 
[ i+, t tel l} 
This integral may be accurately evaluated by (69), providing 8(A,)¢ > | 
to insure that the truncation error at \; is small. The photodensity func- 
tion for a constant electric field defined by (45) can be written 

AG 


: 8 
F(¢,L,6) = al ~ ((1 — o8)] dd, (71) 
¢ ] pt ¢ a a + pp 70s ( a8)| « ( 


and its derivative is defined by 


he 
) . B os 
F’(¢,L,8) = iad | ~ {1 — Btylp¢(1 — o8)]} dv. (72) 
6,L,8) = pe Bs pe Bel pi 8)]} 72) 


These expressions are in the form of (22) and (70) respectively, and 
can be evaluated by (69) in most cases. 

Some of the integrals occurring in the theory cannot be evaluated 
with sufficient accuracy by (69) because of large contributions from the 
short wavelengths \ < A; = 0.42 uw. As an example of this difficulty con- 


sider 
eG 
| vir) dy = 1, (73) 
0 


which follows from the definition of v(A) in (20) and of N in (2). The 
quadrature (69) gives 0.905 for this integral, which shows that 


0.42 
vdXx = 0.095. 
0 
This result can be used to correct (69) for the evaluation of 


Ag 
F’(0,L) =| r oa 
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occurring in (31). If B(A,) > 1 it is only necessary to add 0.095 to the 
right side of (69); thus for szlicon 


10 


, f 8 
F (0,L) 0.095 + (0.33R;)v 26 
~ TL + B; 


The evaluation of F’(0,L,8) occurring in (50) is more difficult and re- 
quires a numerical integration over the larger region 0 < A < Ag. To 


(75) 


carry out the integration the absorption curve must be known for very 
high absorption® a ~ 10° em‘. There may also be a significant correc- 
tion to be added to (69) in the evaluation of F(¢,,8) for very high fields 
& > 100. These corrections can be readily calculated by integrating 
from 0.22 to 0.42 u by the trapezoid rule. The necessary data for silicon 
is listed in Table III. For large fields (72) becomes 


ha 
,/ ] = ie me 
k (¢,L,8) > — vBe dx, (76) 
gow 26 J0 
‘ . oe a . ope ° 4 
which can be evaluated by (69) if 8(A,)¢ > 1. For silicon with a = 10 
em the following values were found for the integrals appearing in (23), 


(36), (51) and (76): 


| vad = 3.7 X 10° em", 


= 0,754. 


1490 em 


TABLE III 


0.00477 
0.0148 
0.0506 
0.0625 
0.178 
0.306 
0.448 
0.538 
0.654 
0.801 
1.14 
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The Covariance Function of a Simple 
Trunk Group, with Applications 


to Traffic Measurement” 


By V. E. BENES 


(Manuscript received July 6, 1960) 


Erlang’s classical model for telephone traffic in a loss system is con- 
sidered: N trunks, calls arriving in a Poisson process and negative expo- 
nential holding times; calls which cannot be served at once are dismissed 
without retrials. Let N(t) be the number of trunks in use at t. An explicit 
formula for the covariance R(-) of N(-) in terms of the characteristic 
values of the transition matrix of the Markov process N(-) is obtained. 
Also, R(-) ts expressed purely in terms of constants and the ‘“‘recovery”’ 
function, t.e. the transition probalnlity Pr{N(t) = N | N(O) = N}; R(-) 
is accurately approximated by R(O)e"', with r; the largest negative char- 
acteristic value, itself well approximated (underestimated) by —E{N(-)}/ 
R(O). Exact and approximate formulas for sampling error in traffic meas- 
urement are deduced from these results. 


I. INTRODUCTION 


A theoretical study of sampling fluctuations in telephone traffic meas- 
urements is useful both in designing procedures for measuring traffic 
loads and in interpreting field observations. Hayward' and Palm’ have 
given an approximate formula for the sampling error incurred when 
observations of the numbers of calls in existence are made at fixed in- 
tervals of time. Their formula has the disadvantage that it is derived 
for a probabilistic model (of the traffic) in which there is an infinite 
number of available trunks. Thus there is no limit to the number of 
calls which can be in progress at one time, and no congestion. Two im- 
portant parameters, the number N of trunks in the group, and the prob- 
ability py of loss, are left out of account. For this reason the practical 

* This work was completed in part while the author was visiting lecturer at 
Dartmouth College, Fall-Winter, 1959-60. 
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application of this model is usually restricted to large groups of trunks 
which are lightly loaded. 

In this paper we derive and study the covariance function of the sim- 
plest stochastic model of a finite group of N trunks. The sampling error 
in trafic measurements can be calculated exactly from the covariance. 
We find formulas for the magnitudes of fluctuations of observed traffic 
for both periodic and continuous observation. The exact formulas lead 
to simple approximations similar to Hayward’s, which take account of 
the number of trunks. Our results are summarized and discussed in 
Section IT. 

We shall use A. K. Erlang’s classical probabilistic model for a group 
of trunks, described as follows: 

i. Holding times of trunks are mutually independent, each with a 
negative exponential distribution. Time is measured in units of mean 
holding time. 

ii. Epochs at which calls arrive form a Poisson process of intensity 
a > 0, independently of the holding times. The offered load is then a 
erlangs. 

iii. There are N < = trunks; calls which find all N of these trunks 
busy are “‘lost,’”’ and are cleared from the system. 

These assumptions determine a Markov stochastic process N(t), 
—x* <t< «,the number of trunks in use at time ¢t. N(-) is a random 
step-function fluctuating in unit steps between 0 and N. As is well known, 
N(-) has stationary probabilities |p, , n = 0,1,---, N} given by the 
(first) Erlang distribution 


equilibrium probability that n trunks are busy. 


With this choice of absolute probabilities, N(-) is a strictly stationary 
process, whose mean and variance are respectively 


my a(l — pw), 
m, — apy(N — m). 


The probability py of loss is shown in Fig. 1, the fractional occupancy 


r—] . - 4 ° 2. - P 
N m, in Fig. 2, and the variance o° in Fig. 3. 








° 2 
b w 


Py, PROBABILITY OF LOSS 
? 
° 
Ww 

















2 3 
@, OFFERED TRAFFIC IN ERLANGS 


Fig. 1 — Probability py of loss. 














4 5 
4, OFFERED TRAFFIC IN ERLANGS 


Fig. 2 — Fractional occupaney m,/N 
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a?, VARIANCE (=R(0)) 
= = id ed Nv nd 
oa @ °o N a oa 


‘ 


3 4a ) 6 
&, OFFERED TRAFFIC IN ERLANGS 


Fig. 3 — Equilibrium variance. 


II, DISCUSSION, SUMMARY AND CONCLUSIONS 


The covariance R(t,s) between samples N(t),N(s) of the stochastic 
process N(-) is the average of the product of N(t) and N(s), minus 
the product of the averages: 


R(t,s) = E{N(t)N(s)} — E{N(t)} E{N(s)}. 


Since N(-) is a stationary real process, we have R(t,s) = R(|t — s| ). 
The function R(-) is called the covariance function of the process N(- ). 





COVARIANCE FUNCTION OF SIMPLE TRUNK GROUP 


It can be written as 


R(t) = lim E{N(t + u)N(u)} — BE{N(t + uu) E{N(u)} 


ur-eo 


m=0 n=0 m=() 


N N N 2 
=> MDm bs n Pr{N(t) = n| N(O) = m} — (> mpm) ‘ 


where {p»} are the stationary (or equilibrium) probabilities given by 


(1), and 
Pr{ N(t) =n | N(O) m} 


denotes the transition probability that n trunks are busy at time ¢ if m 
were busy at time 0. The function R(-) expresses the average depend- 
ence or correlation between samples of N(-) taken at times ¢ apart. 

The principal practical use of the covariance function R(-) in the 
theory of telephone traffic is in computing theoretical estimates of sam- 
pling error incurred in traffic load measurements. Two methods of meas- 
uring traffic, the switch-count and the time-average, are considered in 
this paper. In the switch-count, n observations {x ,---, %,,2; = N(jr), 
j = 1,---,n} of the random process are made at aunvale:s r apart; the 
average 


. >> N( jr) = Ly 2 = 


n j=1 L j=l 


is then used as an estimate of the carried load m, = a — apy. This 
method is important economically because it is cheaper to scan trunk 
groups periodically than to observe them erences: + The number + 
is the scan interval, and the number S, = 2, + --- + 2, is called the 
(total) number of paths in service, in n observations. Table I lists actual 
mean holding times, scan intervals used and resulting values of +r for 


TABLE I — Houpine Timgs, ScAN INTERVALS AND VALUES OF T 
: 


| Ratio r of Scan 
Interval to Holding 
‘ime 


Scan Interval 
Typical (seconds) 
Type of Call Holding Time 
(seconds) = 


U.S.A. | Europe | U.S.A. Europe 


Local Call | 100-200 | 100 =| ~=—36 1 to} | } to} 
Long Distance Call | 200-600 | 100 | 36 4 to 
Originating Register Holding 10-15 10 or 100 36 1 to 3 
Time or 
10 to 
o. 5 Marker Holding Time | 0.25-1.0 10 - 
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‘ » _— . . la ° P " 
various types of call. The variance of n ‘S, is expressible in terms of 
the covariance R(-) as 


™ lea ‘ ° . 
Varin S,,} — |j|)R( gr). 


In the ééme-average, the continuously recorded sample average 


~T 
M(T) =T"| N(t)dt, 
“0 
is used to estimate the carried load. The variance of this estimate is 
~T 
Var{M(T)} = 27" | (1 — t)R(t) dt. (3b) 
Jo 
Thus the mean square error of both these methods of traffic measure- 
ment can be calculated theoretically if the covariance R(-) is known. 
In formula (2) the covariance function is expressed in terms of the 
stationary probabilities {p,} given by the Erlang distribution, and the 
transition probabilities 


Dmn(t) Pri N(t) = nj N(O) = m}. 
In the theory of telephone traffic, the particular transition probability 


pyn(t) = PriN(t) = N|N(O) = N 


9 


has been singled out (in Refs. 3 and 4, for example) as a suitable ‘“‘re- 
covery” or “relaxation” function that is characteristic of the dynamic 
behavior of the Markov process N(-) in point of the undesirable ‘all 
trunks busy” condition. 

We shall show that a much more cogent reason than this can be ad- 
duced to support the importance of the recovery function to traffic 
theory: the covariance function R(-) can be expressed entirely in terms 
of the recovery function and the offered load a. In other words, a single 
one of the (N + 1)° transition probabilities appearing in formula (2) 
suffices for determining the covariance function, and this one is the 
recovery function pyw(-). This fact is a theoretical justification of the 
intuitive view that the recovery function is important, for now the vari- 
ances of n ‘S, and of M(T) are expressible using only the recovery 
function. 

We next give a summary of the contents of the remaining sections; 
this is followed by an account of specific results and conclusions. 

An exact formula for the covariance R(-) is stated and discussed in 
Section III, and derived in Section VII. The formula readily yields a 
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rigorous upper bound which appears to give a close approximation to 
R(-) itself. In Section IV the recovery function pywy(-) is given, and it 
is shown how the covariance may be expressed in terms of the recovery 
function by a convolution integral. The variance of n‘S, is studied in 
Section V; an exact formula, and an approximating upper bound [based 
on the upper bound for R(-)], are both obtained. The variance of the 
time-average M(T') is considered in Section VI; again, an exact formula 
and an approximating upper bound are found. 
The covariance function R(-) is bounded from above and closely 
approximated by a single exponential function 
R(t) < oe", o = R(0), r < 0. 
Here 
o = R(0) 

= equilibrium variance of N(- ) 

= (load carried) — (load lost)(average number of idle trunks), 
and the reciprocal time constant 7; in the exponent is the dominant* 
characteristic value of the “rate” or ‘‘transition’’ matrix of the differ- 
ential equations satisfied by the transition probabilities. Alternately, 


r, is the root of least magnitude of a Poisson-Charlier polynomial. The 
root r; is shown as a function of offered traffic a for N = 1,---, 8 in Fig. 
4, and is tabulated in Table II. 

A lower bound for r; , depending only on the mean and variance of N(-), 
is derived in Section VIII by making use of the fact that the matrix of 
the differential equations for the transition probabilities is symmetriza- 
ble. For low values of offered traffic per trunk, i.e.,a/N < 1, this bound 
can be used to approximate 7; . In any case, the bound is a convenient 
starting place for the use of Newton’s method. The bound is the ratio 
—m,/o°, which satisfies the inequality 


my, 
= wf, < —l, 


= 
with 
my equilibrium mean of N(- ) 
= load carried = a(l — pw), 
= equilibrium variance of N(- ) 
= (load carried) — (load lost)(average number of idle trunks). 


The approximation r; = —m,/o" is illustrated in Fig. 5. 


* T.e., that of least magnitude (among the nonzero characteristic values). 
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2.6 





| 
| 
}— sy th 














Fig. 4 — Negative of the root r; of smallest magnitude as a function of load a 
for N = 1,---, 8. 


TABLE II — NEGATIVE OF DoMINANT CHARACTERISTIC VALUE 11 


a = 4 | N=5 | N=6 N=7 N=8 

.000062 
.004800 
.037229 
. 121762 
. 265214 
.463798 
.710891 
.000000 
.325514 
682770 


.043967 | .011448 | 
. 249464 . 112166 

.§82363 | .326321 
.000000 .§29624 
.477548 | 2.000000 
.000000 | 2.422137 
557618 2.885474 
143703 3.382497 
. 753426 3.907677 
.383178 .456828 


.002421 | 
.045044 | 
172257 
. 383389 
. 663799 
.000000 
381627 
800900 
251918 
730121 


BD BND ND Oe 


OU He 09 09 BD ND et et ee 
CO G2 ND DDO 
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Fig. 5 — Illustration of the approximation ry  —m;,/e?. 


By the “infinite trunk’’ model we shall henceforth mean the stochas- 
tic model for telephone traffic determined by all the same assumptions 
that we made in the Introduction, except that NV = «;i.e., an unlimited 
number of trunks is postulated. Riordan’ and Bene’*’’ have considered 
this model; Hayward’ based his sampling error formula on it. 

It is widely believed that the “infinite trunk”’ model is applicable to 
large groups of lightly loaded trunks. Such a belief is gratuitous until 
comparisons with a model having a finite number of trunks are made. 
Studying the covariance function of the simple finite trunk group en- 
ables us to make some of the needed comparisons; e.g., the variances of 
n 'S, and M(T’) in the two models are of particular interest. Knowledge 
of the covariance R(-), however, is also relevant to the other three cases 
to which engineers are loath to apply the “infinite trunk” model, viz.: 


i. large groups of heavily loaded trunks, 
ii. small groups of lightly loaded trunks, 
iii. small groups of heavily loaded trunks. 
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rr ° . lia = . . 
The variance of n ‘S, is bounded from above and approximated by 
the formula 


esch’ )) 


, lo 2 — 
Varin S,'} Sn o <ectnha — = 
2n 


where n is the number of observations, and 


—+# (sean interval) (dominant characteristic value). 


The exact formula for the variance of n-'S,, in the “infinite trunk”? model 


1S 


n ‘a <ctnh jr : = esch* AT. (5) 
all 
The upper bound (4) for the finite group is compared with the exact 
formula (5) for the “infinite trunk”? model in Fig. 6, which shows each 
formula as a function of the scan interval 7 for various n, for a = 20 
erlangs offered to 20 trunks. The curves suggest that the upper bound 
for Varin 'S,} for N < is consistently less than the corresponding 
variance in the “infinite trunk” model. As might be expected, increasing 
the scan interval 7 improves accuracy for the same number of observa- 
tions. This is because the covariance function is positive, and monotone 
in | ¢ 
The variance of M(7') is bounded from above and approximated by 
rT m 
Var{M(T)} < 2027@ 1 — 0 (6) 
= Tr? 
where 7 is the length of the time-interval of continuous observation, 
and o and 7 are, as before, the variance of N(-) and the dominant 
characteristic value, respectively. The exact formula for the variance 
of M(T) in the “infinite trunk’”’ model is 
tuk (7) 
i 
Since r, < —1, and o is always less than a if N < «, the “infinite 
trunk” model overestimates the variance of M(7’) if applied to a finite 
group. This conclusion is illustrated in Fig. 7, which shows the formulas 
(6) and (7) for a load of 20 erlangs offered to 20 trunks. For an observa- 
tion time of 10 mean holding times the “infinite trunk’’ formula (7) 
applied here would overestimate the variance by about 500 per cent. 
This is about as extreme a case as would occur in practice. Fig. 7 also 
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Fig. 6 — Comparison of variance of S,/n for finite and infinite trunk models. 


depicts a “mixed” formula obtained by replacing a by o° in the “infinite 
trunk” formula (6); for 10 mean holding times the “mixed” formula 
only overestimates the variance by about 100 per cent. Thus most of 
the discrepancy is due to the difference between o° and a. 

Our conclusions are set down in the following list: 

1. The average dynamic behavior of the process N(-), as described 
by the covariance function R(-), can be adequately determined from 
the dominant characteristic value r; and the variance o of N(-). 

2. The same parameters, r; and o°, suffice to give simple approximat- 
ing upper bounds for the sampling error incurred in both periodic and 
continuous observation of N(-). These bounds depend on the size N of 
the trunk group. 
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Fig. 7 Comparison of variance of M(T) for finite and infinite trunk models. 
3. In terms of r; and o° it is possible to check the applicability, for 
theoretical estimates of sampling error, of the “infinite trunk” model 


which assumes N = ~., 
4. The “infinite trunk” model, applied to finite trunk groups, con- 


sistently and often grossly overestimates the sampling error. The over- 


estimation occurs largely because o is always less, and for heavy traffic 
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is much less, than a, the (Poisson) variance of N(-) in the “infinite 
trunk” model. 

5. In terms of r; and o’ it is possible to design sampling procedures for 
traffic measurement that depend explicitly on the number N of trunks 
in the group. By these methods, a given accuracy can be obtained with 
less observation, and thus at lower cost, than the “‘infinite trunk’’ model 
would require. 

6. Hence for finite groups of trunks traffic sampling procedures which 
are based on the “infinite trunk’”’ model tend to be wasteful, particularly 
for heavy traffic. The parameters r, and o° provide a systematic way of 

tailoring the measurement procedure to the number of trunks in the 
group. 


Ill. THE COVARIANCE FUNCTION 


To state the formula for R(-) we need the “sigma” functions* de- 
. : 9 
fined (see Riordan’) as 


ool m) 
: m k i m—j 
ox(m) = > ( TJ ‘) Pek. al ” 
j=0 J (t= 3) 


with m (but not k) a nonnegative integer. These functions are connected 
with the Poisson-Charlier polynomials 


p(x) = a"?(nt)* > (Ey (i); () 


by the relation 


moony \—4 
on(m) = (—a')"(m!) Dal —k). 


(See Ref. 10, p. 33.) 

For fixed N and a, let r; , 72, «++ , rw be (in order of increasing magni- 
tude) the N zeros in the variable s of the polynomial ¢,,;(.V). In Sec- 
tion VII the covariance is shown to be given by (the exact formula) 


N 
R(t) = —a'px 2, - I]t — (r; — ri) (8) 


j=1 7 r3(1 —- r;)? if) 
where py is the probability of loss. It has been shown that the zeros 
rj are i all real, negative, and distinct; all are less than —1, and consecu- 


’ T he o notation is copied from unpublished work of H. Nyquist. The functions 
themselves were introduced into traffic theory by Palm.’ 

+ The earliest reference appear to be Hs aantjes!" in 1938. See also Ledermann 
and Reuter.’ 
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tive pairs are separated by at least unity. Fig. 8 shows these roots for 
N = 1, 2, 3 as functions of a. 
Now r; is always negative, and the terms of the product satisfy 


ne 
r r; 


3 
hence the sum in (8) has all terms negative, so that 


Rit) = 6. all ¢. 


The correlation between successive samples is thus always positive. It 
is obvious from (8) that 


N 


—a' py ps II 2 ~ “i =o = R(0). (10) 


j= 73(1 + 17;)? ix; 





———— NEGATIVE OF ROOT OF @g,, (1) 


NEGATIVES OF ROOTS OF @,,, (2) 

} 

NEGATIVES OF ROOTS OF @,,, (3) A 
, + - = at 


— TO fH 


/ 
1) 











5 6 
OFFERED TRAFFIC, 


Fig. 8 Roots of the first three o-functions 
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Since 7; is the root closest to zero, the value of (8) is only increased 
if the r; in the exponents of (8) are replaced by 7, . Using (9) and (10), 
we conclude that 


0 s oe" — R(t) = £(t), (11) 
where 


N t ryt 


made ts estas. 


jut PHL SE P3)* ies | RO 


9 


4r° — 30 _, 
€ 


24 


N 
< a’pye' D> (fj +1)? S a’py 
j=2 


< (0.3933)a’pye 


The approximation R(t) & o’e™' is illustrated in Figs. 9 and 10. It ap- 
pears to be fairly accurate, especially for light loads. 

The upper bound o’e"' for R(t) should be compared with the rigorous 
formula (see Riordan’ and Benes’ ) 


R(t) = ae E 


which holds for the “infinite trunk”’ model. In this model the equilibrium 
distribution of occupancy is Poisson, so that 


R(O) = o = Var{N(t)} = E{N(t)} = a, 


and the “time constant” of the exponential is unity, since time is meas- 
ured in units of mean holding time. 

The difference between the ‘infinite trunk”? model and the “finite 
trunk”? model in point of the covariance can be understood by con- 
sidering the effect of congestion, which is present in the latter. Conges- 
tion affects the upper bound formula most directly through the value 
of the variance o°. It is obvious intuitively, and borne out in Fig. 3, 
that as a increases o° must eventually decrease to zero. This behavior is 
not mimicked by the “infinite trunk” model, for which o° = a. 

The finitude of N, i.e., congestion, affects the bound o’e'"" in two ways: 
(a) the “time constant” is not unity but the smaller number — (7) 
so that the rate at which dependence between samples of N(-) de- 
creases (as a function of the interval between samples) is larger than 
in the ‘infinite trunk”? model; this ‘‘time constant” decreases as the 
traffic a increases, because, as illustrated by Fig. 4, 7; is a monotone 
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Fig. 9 — The covariance R(t) for N = 5 trunks, a = 10 erlangs, with the ap- 
proximate formula R(t) ~ oe". 


decreasing function of a; (b) the value of R(0) (=o') is not a but the 
generally much smaller number 


o =a(l — Pn) 1 _ aps ( : = i) |, 
a— apy 


= all — py(l + N — a + apy). 


The last form shows that o < a for all a and N. In fact, it is obvious 
intuitively that 


o =m, — apy(N — m) < m < a. 


A simple approximation for the dominant root r; can sometimes be 





COVARIANCE FUNCTION OF SIMPLE TRUNK GROUP 


3.4 





N = 8 TRUNKS 
oe iS a aoe | @=4ERLANGS 
| | | R(t) 
—_ o2zeMmt 
24h} -— ——_—_—- 


3.0 





2.6 


2.4 








2.2 











0.2 0.3 04 05 06 07 08 09 1.0 
t 


Fig. 10 — The covariance R(t) for N = 8 trunks, a = 4 erlangs, with the ap- 
proximate formula R(t) ~ oe"!. 


used to make the approximation R(t) & o’e''"! more useful. It is shown 
in Section VIII that 


my _ sarried load 


a load variance ~ 
i.e., —m,/o is a rigorous lower bound to 7, . Fig. 5 suggests this bound 
gives a fairly good approximation to 7; if a/N < 1. Hence a simple 
approximate formula for R(-), valid for a/N < 1, is given by 


R(t) & oem") 


carried load P 


> (load variance) exp) — (12) 


load variance f 
We know that R(t) < o’e'"" and that —m,/o” < r, ; hence replacing 
r, by —m,/o° tends to correct the error in the upper bound formula. 
The formula (12) is illustrated in Fig. 11. 
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Fig. 11 — Comparison of R(t) with oe for N = 8 trunks, a = terlangs. 





COVARIANCE FUNCTION OF SIMPLE TRUNK GROUP 
IV. THE COVARIANCE IN TERMS OF THE RECOVERY FUNCTION 
It has been shown" that the Laplace transform of pyy(-) is 
a,(N) 
80541 N ) , 
By expansion in partial fractions we find that 
. 


eS t 
Pwn(t) = py — } i I] (1 _ =} (13) 
rj rT; ee 


j=1 ix) i 


The sum assumes Gnly negative values, and so pyy(-) decreases mono- 
tonically to the loss probability py . The recovery function is illustrated 
in Fig. 12. 
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Fig. 12 — Recovery function for N = 5 trunks, a = 10 erlangs 
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We now observe that for each 7 = 


at 


(¢—ue” ''* du = - a s 
Jo (j+1) r+1 


t 


(14) 


By comparison of formulas (8) and (13), and use of (14), one finds that 


at 


a’ Dy (t— ue" [pyy(u) — py] du + o’e * + Cte, (15) 


“0 


Rit) = 


where 
; 2 1 | 
C = —apy > - I] (1 a perees =). 


ju 7317; 1+ ies rj 


This formula expresses R(-) in terms of pyw(-) by a simple convolution. 
To evaluate C explicitly we note that 


. 2 O41(N — 1) a_y 
Cm iy ee ee | 
(1 + s)osui(N) 1 + 8 |s—0 


where a_, is the first coefficient in the power series expansion of the left- 
hand term in the bracket. One finds 


oo N — l } 


ao( NV) 


2 N 
apy (* —1+ ps) 


apy(N — m,) 


(load lost) (average number of idle trunks). 


V. THE VARIANCE OF THE NUMBER OF PATHS IN SERVICE 


We assume that n observations {z;,7 = 1,---, n} of N( 
during an interval of equilibrium, so that 


-) are made 
Coviz;,27;} = R(jt —Jj\ 7), 
where 7 is the scan interval. Then with 
S, = Ty ss v2 + as if 2s a Ln 


= number of paths found in service, 





COVARIANCE 
we find that 


Var} S,} 


To give an exact formula for Var{S,,} 


|m|\u 


9 


2im\u 


oe) 
> (m — n)e?™ 


m=n 


Then also 


n 


FUNCTION 


netnhu — 


OF SIMPLE TRUNK GROUP 


) (nn ) 
y2 ) 
>» — BCD) x) 


} \ i=] 


(n — | j|)RGjr). 


we note that 


= ctnh u, 


a0 
d 2mu 1 2 
€ = sesch u 


du m=1 


m 


«2 
+2)>> (m—n)e™ 
m=n 


(1 ce ane) 


a 


esch’ u.* 


Since the covariance R(-) is a symmetric function given by (8), it can 


be seen that 


r —la 
Var{n S,} = 


Ny | etnn( — 
—n a’ Dy b i 


Tr; 1 — 
2/ °§#2 


nTr 
e/* 9 Tr 
esch*| ——~ 
n 2 , 
(18) 





j=l 


‘II 


ix) 


rj(1 + 7r;)? 
(1-1) 
ri — fi 


* Use of this identity was suggested by unpublished work of J. W. Tukey to 


which the author had access. 
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This formula is exact, given the assumptions. It is easily shown from 
. oy . . . So 2 
formula (17) that the exact formula for the variance of n 'S, in the 
“infinite trunk”? model is 


l 


etnh ir — 


illustrated in Fig. 6.° 

Returning to the case of finitely many trunks, we can obtain ap- 
proximating upper bounds to formula (18) for Var{n 'S,} by using the 
results of Section III on the covariance function. It can be seen from 
the arguments leading to (17) that replacing the roots r; by 7: in the 
hyperbolic functions in (18) increases the values of the expressions in 
square brackets; this replacement is equivalent to using the upper 
bound 


9 
2 ryt 
oe 


for R(t) in formula (16). Hence 


r lo 12 l—e 2 
Varin S,} Sn o <scetnhaA — - esch” \> 


where 


Tr ° ° ee 
9 }(sean interval)(dominant characteristic value). 


Since oe’' is close to R(t), we may expect that the overestimate (19) 
gives a good approximation to the actual variance. This approximation 


is conveniently plotted as a function of \ for various n in Fig. 13. 


VI. THE VARIANCE OF TIME AVERAGES 


It follows from formulas (3b) and (8) that 
aT 


Var N(t) dt} =a + aT + o(e 


70 


as T — x, where 





- I(1-- 1) 
Co = 2a’ py >, = : : 


1 
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Fig. 13 


is a negative constant, and 


C - | R(u) du = 2a°py 


Note that co and ¢, differ only in the power of r; that occurs in the 


0 


Upper bound to Var{S,}/ne?. 


v I] 


2 


| 


(.- 





>. = 


j=1 


re(1+r,)2 


denominators. The third term of (20) is positive; is given by 


ole 


IY . . T 
equals —¢p at T = 0; and is of smaller order than e “ because r; < —1 


") = —2a’py ts 


N 


j=l 


r,7 


e 
r?(1 + 17;)? 


To evaluate c, explicitly, we note that 


qQ= = 2a'pa| 


Os41(N — 1) 


s(1 + 8)?o.1(N) “a 


l 


— Py 


8 


I = - 


rj 


M(i-—.) 


a_\ a_» 





~ - » Ze) 
l +s (1 | 0 
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where a_:, a_; are respectively the first and second coefficients in the 
power series expansion of the leftmost term in the bracket of (21); these 
are given by 


oo( N —_ 1) 


ao(N) a’ 
. d O2r41(N i 1) 
a, =— - 
aa * LOer1(N) jrm-1 
1 — py 
apy 


To find c; we must compute 


lim = nell | ES Se a _ _@  oayi(N — 1) 
0 L8(1 + 8)?o.41(N) 8 dx (1 + x)®oz43(N) Jrmv 


This equals 


d Or41( N — 1) 
2(1 — py) — ; ; 
sha dx Gz4,(N) | 0 


' d é 
(1 — py) | 2 — log gr44(N — 1) + ¢ log o74,(N) 
dx dx 


r=( 
Now the generating function of the o-functions is 


2 


@(s,z) = >, 2"0,(n) = (l—2z 


so that 


___atj) 
5 (nm — j)oi(n) ~ 
It follows that 


¢; = 2a px[a_2 + a. + (1 — pw)(2 — Envy + Ew)] 
= 20° + 2a pw( 1 - pn) — En l + En) + 2aN py ° 


The constant ¢) can be evaluated in a similar fashion. 
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From the bounds (11) for R(-) we conclude that 


eet. nf) — Var{M(T)} 


“ere ap re — 


4 
é 


< (0.3933) a"py (£ ial 


e 2 t . 
and since R(t) = oe", we may expect that the overestimate 


T ’ 
aT ij -l- rl 


2 saicaipdeisiaapagi (99) 
<0 rr). ¢ ~~ 
Tr,? 


is a good approximation to the variance of M(T7T). This approximation 
has the same form as the exact formula (7) for the “infinite trunk” 
model, because in both cases a single exponential is used for R(-) in 
formula (8). The overestimate (22) is depicted graphically in Fig. 14. 
It was convenient to plot the ratio 
Var{M(T)} 

o 

as a function of the single parameter 
uw = 17’ = —(dominant characteristic value) (observation time). 
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A simpler form of (22), valid for a/N < 1, results when we replace 7 
by its lower bound 


m, carried load 


9 


a ~ load variance 
This replacement decreases the value obtained, i.e., moves the approx- 
imation in the direction of Var} M/(T)}. 
VII. DERIVATION OF THE COVARIANCE 

The transition probabilities 

Dmn(t) = Pr{N(t) = n| N(O) = m!} 

of N(-) satisfy the Kolmogorov equations 
Pmn(O) = dmn; 


d 
d 


d 
d 


1 Pms - ADP m (Nn + is NDmw ’ 


Dan = (N+1) Panga) FADm(n—1) — (A +N) Dan; 0O<n<N, 


t Dy _ Don ap. 
nO Pmi Pm0 - 


Multiplying the nth equation by n, and summing on the index n, we 
find 


tf 


ee . ier : 
y iN N(O) = m} = —E{N(t) | N(O) = m} + a{l — pan(t)], 


tf 


whence 


’ , , t ( u 
E\N(t) | N(O) m} me +a | e "TL — pay (u)) du. 
“(0 
By formula (2), the covariance is then 


N 


Rit) = . i mMpmE{N(t) | N(O) = m} — m), 


m=) 


N 


t 
t t 2 (t—u 
Moe +am(l—e )—m —-— af e > MPmPmw' u) du, 


0 m=() 
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where 


N 


m; = >, n'p, 


n=() 


for i = 1, 2 and {p,} are the stationary probabilities given by (1). 
particular, 


m, a(l — pw), 
o (mz — m,°)' = [m, — apy(N — m)|’. 
The Laplace transform of 
Pr{N(-) = N|N(0O) = m} 
has been determined’ to be 
a’ "m!o,(m) 
N!s0.41(N) 
Therefore that of R(-) is 
R*(s) = [ e"'R(t) dt = 4 am my 
0 IlI+s s(1 +s) 8 


(26) 
N 


N-—m ' 
a = a mio,(m) 
— = MPm - ; 

8(1 + 8)os41(N) m=1 N! 


By (1), the last term of (26) is 
ap ~ 
— ; mo,(m). 
s(1 + 8)oe41(N ) m=1 
It has been shown’ that the “sigma’”’ functions satisfy the recurrences 


o.(m) = oe41(M) — on4i(m — 1), 


ma,(m) = ao,(m — 1) + S8oe4i:(m — 1), 


so that 


N N-—1 N-—1 
> ma.(m) = ad, o.(k) + 8 >> asyi(k) 
m=1 


k=0 k=0 


= dosai(N — 1) + Son42(N — 1), 


o42(N — 1) N aouyi(N — 1) 


o:41(N) ” s + l (8s + L)ossi(N)- 
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The foregoing identities yield the following simplified formula for R*(s): 


R*(s) = Ms es am, - aN py 


l+s s(1 + s) (1 + s)? 


a apy Os4i(N — 1) | 7 m, 
1+ sLs(1 + s)ouyi(N) s 


From (27) we find that the partial fraction expansion 


(29) 


o,,(N)N! 


O244(N = 1) - = we 
O241(N ) 2, (s—r;) [] ( - 


1) 


N 
= Zz. a == 99) 
j=l 


is valid, where {r;} are the zeros of o,4;(N). 
By a similar argument, since py = oo(N)/oa\(N), 


Gu(N—1) —_1— pp am N 


S(1 +. 8)osa,0N ) s a(l re s) 


Pr Tj ix) 
Hence formula (29) can be inverted to give, for ¢ = 0, 


t t r t 2 
R(t) = me + aml — e | — aNpyte" — m, 


at r ju 


. \V dl e”? 
— a'pe | , | 1 - ps 7 a° are ey 


“0 


(30) 


tet lay, 
rj 


Fz Sf 


N rot 


=ge + a pyKe ° a py : :.. - I] < toad St iF 


mi Ti(l + rj)? iz; ri — 7; 


where 


"= mM. — m = equilibrium variance, 


oo. eee rey 
j=el 


(Cl + 7;)* i435 975 — 48; 
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To evaluate K explicitly we observe that 


es Ou(N—1) ay G4 (31) 
(1 + 8)os41(N ) l ot 8 (1 + s)? 8 0 aes 


where a_2, @_; are respectively the first and second coefficients in the 


power series expansion of the leftmost term in the bracket of (31). Thus 
K =a, +a2—1+ py. 
Now 


Os41(N — 1) ga l 
. =: {} s) (1 Ss) 
(1 + s)?o.4i1(N ) iis Fy + v 


d @r4,(N — 1) aa (ge — #;) L—3=s 
; +1\ 1 4 E edl- j TL’ r ! 
dx daui(N) 1 jut (1 + 23) i; 1; 
From the recurrence age for the o-functions we find that 
, Os 11 ( N Os ( N 


= @- 
aN) ai. ete 5 


differentiating with respect to s and setting s = 0, we obtain 


d o.41(N — 1) | ( oi(N _ = _ Dn 
a , = — = = intl. salient - =—_- — ‘ 


ds 341 N ) 8 a ao N) apn 


Clearly, 


and so 
¥ q N N 
K = —-—_—_“+— —1+ py, 
a pyK 


Thus the formula (30) for the covariance function R(- ) simplifies to 


NV r,t 


R(t) = —a’py pa II # — (32) 


+ 5)? ij 


VIII. APPROXIMATION TO THE DOMINANT CHARACTERISTIC VALUE 
The differential equations (23) can be written in the form 


d 


(4) = QOPi8), 
at Q ) 
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where P(t) is the matrix of transition probabilities {p,.,.(t)} and Q is 
the matrix of the ‘‘transition rates’’: 


a 


0 


The characteristic values of Q are 0, 7; , 72, +++ , ry . We define 
1 «ee : 
Wn = —=nla")>. -, = 0,1,:--,N, 


Pn id J! 


and we introduce an inner product for the space Lo(u) of (N + 1)- 
tuples of complex numbers by the definition 


N 
(x,y) = Qo tnGnbtn - 
n=() 
The matrix Q represents a symmetric operator on L» 
(Qr,y) = (2,Qy), xy © Le(p). 

It is easily seen that 

N | N 

LY —=L p=, 

n= [bn n=0 

Qp = 0, for p= (po,M, 
Qi jm: (jim; , 1j = 0,1,---,N. 


The last identity implies that 


1 Qj 
(Qzy) = —5 2 (Yibi — Yiej) : (mits — mjr;), 


(Qr,7) =O, 


so (as we already know) all characteristic values of Q are nonpositive, 
being of the form (Qz,x) for some «© Le(p). 
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From the extremal properties of the characteristic values of symmetric 
14 909 TN ‘ 
operators (e.g., Zaanen, p. 383, Theorem 3) we conclude that 


ry = max(Qz,2), 


the maximum being over all «¢ Lo(u) which are not identically zero, 
and satisfy (27,7) = 1, (a,p) = 0, p being the vector of stationary 
probabilities, as in (34). 

We can now estimate r, from below by choosing an appropriate 
vector x. We choose 

ioc 2m @)--- wv 
Thn 

where m, and o are the mean and standard deviation of N(-) in equi- 
librium, given by formulas (24) and (25) respectively. Clearly, (2,7) = 
1 and (2,p) = 0, and 


et n—m n+1l1—m\ 
(Qz,x) = —a z. Dn ( —_ ) 
o Oo 


n=(0 


_ al 1 — py) 


‘ i 15 
(See Kramer. ”) 
This approximation is illustrated in Fig. 5. 
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Mode-Conversion Filters 


By E. A. MARCATILI 


(Manuscript received July 11, 1960) 


Resonance of higher-order modes in waveguides can be advantageously 
used to make band-rejection filters of unusually low loss and simplicity. 
The region where the resonance takes place can be obtained either by a local 
change of cross section of the waveguide or by the inclusion of dielectrics. 
Mode-conversion band-rejection filters can be combined to build channel- 
dropping filters which are of particular interest in the millimeter wavelength 
region to separate bands of TEy° into TE WW. 

In this paper the necessary design relationships for channel-dropping 
filters using mode-conversion band-rejection filters are derived. It also con- 
tains a theoretical derivation of the intrinsic Q of band-rejection filters in 
round and rectangular waveguides. Finally, the experimental results ob- 
tained with different mode-conversion band-rejection filters at 12 and 56 
kme, and with a channel-dropping filter from TEq° to TE\° at 56 kme, are 
given. 


I, INTRODUCTION 


A large variety of channel-dropping filters operate through the use 
of band-rejection filters, and since the microwave art is pushing the 
usable spectrum to higher and higher frequencies, low heat loss and 
easy-to-build band-rejection filters are important. 

This statement is particularly true in the process of separating bands 
in the long distance waveguide communication system’ that operates 
with circular-electric mode in the millimeter wavelength region. The in- 
formation sent from repeater to repeater through the low-heat-loss 
multimode circular waveguide must be separated into tens of bands for 
the purposes of regeneration and amplification at each repeater. Since 
ach repeater operates in single-mode rectangular waveguide, one pos- 
sible solution is to convert the circular-electric wave to fundamental 
mode in rectangular waveguide and then to drop the different channels 
with known techniques. This solution has several disadvantages: the 
filters are relatively lossy because of the low intrinsic Q of parallelepi- 
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ped-shaped cavities in the millimeter region, and they are difficult to 
build. Furthermore, the channels to be dropped last are substantially 
attenuated because they must travel in a high-loss rectangular wave- 
guide. For instance, the theoretical attenuation in the standard silver 
waveguide RG98U (50 to 75 kmc) is 0.53 to 0.39 db/foot. 

Better solutions are filters that simultaneously drop the channels and 
make the transfer from circular-electric wave in round waveguide to 
dominant mode in rectangular waveguide.” 

This paper describes a channel-dropping filter that combines all the 
desirable features: it filters, it transfers TE ° mode into TE”, it has 
low insertion loss and it is extremely easy to build. All this is possible 
because of the use of mode-conversion band-rejection filters. 


Il. DESCRIPTION OF THE MODE-CONVERSION BAND-REJECTION FILTER 


Consider, for instance, a round waveguide carrying the TE° mode 
and barely cut off for the TE.°. If for a length / the diameter of the wave- 
guide is slightly larger, in such a way that TEp° is no longer cut off, 
the region | becomes a multimode region* where the TE92° generated at 
both diameter discontinuities can resonate and introduce a large inser- 
tion loss to the incident TE»,°. The bandwidth depends essentially on 
the amount of mode conversion (size of the discontinuity), and the 
center frequency depends on the length /. The filter can be made of slid- 
ing coaxial tubes because the circumferential cracks do not interrupt 
the conduction current of circular electric modes. 

The intrinsic Q is very high, one order of magnitude better than a 
TE? cavity, because: (a) the resonant mode is essentially a low-loss 
one, (b) the end walls do not absorb energy since they do not exist,* 
(c) the tuning mechanism is lossless and (d) the coupling that is pro- 
vided by the diameter change does not create high-density currents, such 
as exist in the case of band-rejection cavities coupled through irises to 
the main waveguide, or in the case of microwave band-rejection filters 
made of lumped elements. 

The reasoning used for circular-electric modes can be generalized, that 


is, any waveguide that contains a low-loss, multimode region exhibits 
5,67 


rejection bands corresponding to the resonances of the confined modes." 
Thus, a rectangular waveguide cut off for TE2°?, except for a length 
l of slightly larger width capable of generating and supporting TK29~, 


becomes a mode-conversion band-rejection filter. 


* The heat loss due to the penetration of the TEo2° mode in the cutoff wave- 
guides is calculated in Section IX 
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Another band-rejection filter in rectangular waveguide is obtained by 
building the multimode region with a dielectric slab close to one of the 
narrow walls, since the dielectric provides an apparent width increase 
of the waveguide. 

All these mode-conversion band-rejection filters have small return 
loss out of resonance. 


Ill. DESCRIPTION OF THE CHANNEL-DROPPING FILTER 


The channel-dropping filter consists of a through waveguide with two 
multimode regions, one of which is coupled to another waveguide (Fig. 
1). For the purpose of fixing ideas, we imagine that the through wave- 
guide is circular with two enlarged regions where the TE,2° excited by 
the incident TEa° can resonate; the dropping arm is a rectangular 
waveguide. The enlarged regions where the TE»2° mode can resonate 
will be referred to as cavities, even if they are not enclosed volumes. 
The idealized filter must be such that the incident TEo:° mode is matched 
at all frequencies, and at midband all the power flows into the rectangu- 
lar waveguide. 

A low-frequency channel-dropping filter that will be demonstrated to 
be the equivalent of the microwave one and that satisfies the previous 
demands is shown in Fig. 2. The resonant circuits are equivalent to the 
cavities, and the three resistances connected to the circuit through ade- 
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Fig. 1 Microwave mode-conversion channel-dropping filter. 
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Fig. 2 — Low-frequency channel-dropping filter. 


quate transformers (not indicated in the figure for simplicity) are equiv- 
alent to the characteristic impedances of the three microwave ports. 

In each resonant circuit fy is the midband frequency and the loaded 
Q is defined Q, = fo/(2Afo), where 2Afo is the half power bandwidth of 
the dropped channel. The normalized reactances are such that the im- 
pedance seen toward the right of the plane Aa is unity at all frequencies, 
provided that Ya is an odd number of quarter wavelengths. At midband 
frequency the maximum power available goes to R, and far from reso- 
nance it goes to R, . 

Another microwave circuit equivalent to that of Fig. 1, which may 
help the reader to understand the behavior of the mode-conversion 
channel-dropping filter, is shown in Fig. 3. Here, the resonant cavities 
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Fig. 3 — Microwave channel-dropping filter; path difference = 6 . 





MODE-CONVERSION FILTERS 


have been separated from the through waveguide. The incident mode 
excites each cavity through two coupling holes; in an equivalent way, 
in Fig. 1, the incident TEm° mode couples to the TE2° of each cavity 
through two diameter discontinuities. Finally, in Fig. 3, only the reso- 
nant mode of the first cavity is assumed to couple to the output load, 
implying that the coupling between the incident and the branching mode 
in Fig. 1 is negligible. 

The reader who is not interested in the mathematical treatment of 
the mode-conversion channel-dropping filter may now go directly to 
the résumé of results in Section VII of this paper. The scattering matrix 
of the branching cavity, Fig. 1, is studied in Section IV. In Section V, 
the scattering matrix of the rejecting cavity is derived from that of the 
branching cavity by reducing to zero the coupling between the round 
and rectangular waveguides. Then, in Section VI, both cavities are con- 
nected through a certain length of single-mode waveguide, and the math- 
ematical description of the channel-dropping filter is completed. 

The derivations have been made for single-resonance rejecting and 
branching filters because these are the building blocks for the design of 
more complicated filters such as those of the maximally flat type.’ 


IV. SCATTERING MATRIX OF THE BRANCHING CAVITY 

Consider the branching cavity of Fig. 1, separated from the rejecting 
cavity and with all terminals matched. This cavity is represented in 
lig. 4 with the elementary components separated. The symbol J; repre- 
sents the junction where port 1’ carrying TE,0? mode couples symmetri- 
cally to TEn°® and TE,° in the cavity. Since the TEo2° mode is almost 
at cut-off and close to resonance, the coupling to TE:° can be neglected. 
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Fig. 4 — Branching cavity with elementary components separated. 
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B 


Fig. 5 Branching cavity. 


The symbol J. represents the junction at a diameter discontinuity; 
ports 4 and p carry TEq° and port m carries TE 2. Ports M and 2 are 
connected by a piece of waveguide whose ports are N and 5. This wave- 
guide has a midband electrical length 


Y20 a tly 


2 dhe 

where /, is the distance between diameter discontinuities and J, is 
the midband TE 2° mode guided wavelength. Likewise, ports P are con- 
nected by two pieces of waveguide, each of midband electrical length 


¥i0 _ 


tl, 
2 Agi 


’ 


where \,; is the midband TEq° mode guided wavelength in the cavity. 

Fig. 4 can be simplified by representing all the elements inside of each 
of the dotted lines as a single junction J, and so the branching cavity is 
reduced to the circuit shown in Fig. 5. 

Since this three-port structure is symmetric with respect to the plane 
BB, the scattering matrix can be derived from the scattering matrices 
of two simpler structures, one derived by making the symmetry plane 
BB a magnetic plane (open circuit), that is, a surface where the tangen- 
tial magnetic field is zero, and another obtained by making the symmetry 
plane an electric one (short circuit), that is, a surface where the electric 
tangential field is zero. 


4.1 Open-Circuited Half of Branching Cavity 


Assume in Fig. 5 that BB is a magnetic plane. Port 1’ as well as the 
branching cavity is divided in two symmetrical portions, and each half 
is shown in Fig. 6. 
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a, 





“a |MAGNETIC WALL = OPEN CIRCUIT 


Fig. 6 Open-circuited half of branching cavity 
If a; = 1 is the only wave incident in the structure, the outgoing 
waves from the junctions J and J; are 
Pi + aD, 
Pie + aD 2, 
beol'ss + az3I35 , 
bel45 + 3134 ’ 
a3V'33 + bel'ss , 


where I’,,, represents the scattering coefficient between ports mand n. 


From these five equations the scattered waves 


P17T'ss 
(: — Teel's5 + 3) (1 — Ts) — Ts 
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are derived. These expressions can be simplified by assuming 


| I} | 
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and 

Te. 

| Ts | <1. (9) 

| Iss 
The first assumption means that the TE :° mode incident on the diam- 
eter discontinuity has negligible reflection in the same mode. This is a 
familiar approximation in multimode waveguide calculations, and it will 
also be seen later that these reflections are indeed negligible. The second 
assumption implies that the resonating mode TE,:° is loosely coupled 
to the other modes TE»,° and TE,o°. 

Substituting (8) and (9) in the conservation of energy relations® 

applicable to junctions J and J, of Fig. 6, and neglecting terms of higher 
order than two, one obtains the following results: 


= (1 — | T, |’ i (10) 


(11) 


(12) 


(13) 


in which 
—Wi + 035 — O45 + 34 (14) 
and 
g = —¥2 + O22 + O55, (15) 


where y; and yw are the electrical distances between the branching cav- 
ity discontinuities in terms of the nonresonating and resonating modes, 
respectively, and @,,, is the phase of the scattering coefficient between 
ports m and n when the waveguides are reduced to zero length. 

The physical meaning of @ and ¢ will be given later. 

Substituting (8), (10), (11), (12) and (13) in (6) and (7), leads to 
the simplified scattered waves 
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1 + e 
bs = Tyol 45 Tis | * (97) 
if — | Ts | +e”) — —® Je 





») 


The values of 6 and ¢ given in (14) and (15) are frequency-sensitive, 
so we define 


6 = 0 + Aé (18) 
and 
¢ = got Ag, (19) 


where % and ¢o are the values taken by @ and ¢ at midband frequency 
fo, and A@ and Ag are their small departures when the frequency is 


f ae fo + Af (20) 
and 


fo 
In this paper we choose to have the branching cavity resonating with 
an odd number of half wavelengths. In order to have the branching 
cavity resonating with an even number of half wavelengths it would be 
necessary to make resonant the short-circuited half of the branching 
cavity. 
Resonance of the branching cavity (minimization of the reflected 
wave b,) occurs at midband f = fy when the following relation is satis- 
fied: 


AS ey, (21) 


¢o = | Tas | sin 0 — 2ns, (22) 


in which sg is an integer. 

If one substitutes (18), (19) and (22) in (16) and (17), and again 
neglects higher-order terms, the scattered waves of the branching cav- 
ity become 


—idy + | T35|°(1 + cos &) — = 


6 2 
b = ee 


—idy + | Ts |?(1 + cos @) + Pi: 





1 + ef 





bs = Tyol ss 
—iAg + I'35 * l + cos Ay) + 
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Several important results are deduced from the last three equations: 

(a) The resonance condition, (22), as well as the scattered waves, 
(23) and (24), depend on the angle 4, which, according to (14) and 
(18), is the midband electrical length difference between the two possible 
paths that the waves may follow. These two paths are shown in both 
Figs. 3 and 7. The last figure is a reproduction of the branching cavity 
from Fig. 1, and the two paths, as well as the modes with which they are 
measured, have been indicated in it. If 4 is an odd multiple of z, the 
two waves cancel each other and there is no transmission through the 
cavity in spite of its resonance. 

(b) The transmitted and reflected waves, (23) and (24), depend on 
Ay, which is the change with frequency of the electrical length y. of 
the distance between diameter changes measured in the resonant mode 
TE..°. But the same scattered waves are independent of Aé@, which is 
the change with frequency of the electrical length 4. 

Now it is possible to express the scattering coefficients of the open- 
circuited half branching cavity matrix, 


| Su Su 


in terms of b; and by. 
From the definition of scattering coefficients it follows that 
Su = b, ’ 
Su = bs 
and from the conservation of energy relations’ that 
Su* Sig + Si* Sa 0, 


where S* is the complex conjugate of S. 
From (26), (27) and (28) 
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Fig. 7 Branching cavity; path difference = 0, . 
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ELECTRIC WALL = SHORT CIRCUIT 
Fig. 8 — Short-circuited half of branching cavity. 


4.2 Short-Circuited Half of Branching Cavity 


Consider again the branching cavity equivalent circuit in Fig. 5, and 
let BB represent a perfectly conducting surface. One of the halves of the 
bisected circuit is shown in Fig. 8. A unitary wave fed in port 4 yields, 


because of the absence of resonance 


b= -e 


The scattering matrix is S = b. 


4.3 Scattering Matrix of the Branching Cavity 


lig. 9 represents the branching cavity. Considering symmetry 
reciprocity, the scattering matrix is 


! ’ y Y ! 
Ses Sez Ses 

. . 
Sez S77 


| Ses S67 66 | 


All the seattering coefficients in (31) are determined as follows: 
£ 


‘ 
b; 





Fig. 9 Branching cavity with waves fed in phase. 


(30) 


and 





160 THE BELL SYSTEM TECHNICAL JOURNAL, JANUARY 1961 





Fig. 10 Branching cavity with signal fed in branching arm. 


Feed unit power into ports 6 and 8 such that the phases of the electric 
fields are the same. The plane of symmetry becomes a magnetic plane, 
and the scattered waves derived from (26), (27), (29) and (30) are 


be = See + Seg = —b,%e° M12, (32) 
bz = 2Se67 _ VJ/ 2b. 
If unit power is fed into port 7, of Fig. 10, the reflection is 
by a S77 = bh. 
Finally, if port 6 and 8 of Fig. 11 are fed 180° out of phase with unit 


power, the plane of symmetry which has zero tangential electric field 
becomes a short circuit, and the scattered wave is obtained from (30): 
iv, +82034 


, Y 
be = Ses — Seg = —e 


(35) 


Substituting the explicit values of b; and by given in (23) and (24), in 
(32), (33), (84) and (35), and, solving this set of equations for the 





haseal 


Fig. 11 Branching cavity with waves fed in opposite phase 
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scattering coefficients of the branching cavity, one obtains 


| T'35 |? ( L + cos 4) 


1 (263 by) 
=e 34-¥1 





‘hel, ; : ; 
—idy + | A 12] + | T35 |" (1 + cos @) 
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, ’ | f - N 
i(g/2+034—¥1/2+01 1/2) Iy2 | | U's | (1 + cos 4)! 
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—1Ag + - “ + | Tss | (1 + cos 6) 
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- + | I"35 [*( l + cos 4) 
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ae 
—idy — "+ + | Ts (1 + cos A) 


1011 
C 





[Tx |? > 
—ide + E+ + | P35 |? (1 + cos 6) 
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’, SCATTERING MATRIX OF THE REJECTING CAVITY 


The elements of the scattering matrix of the rejecting cavity 
See Ses | 
5 6] 
Ses See | 


| 
| 
| 
| 


(40) 


can be deduced from those of the branching cavity (36) and (38) by 
eliminating the coupling to the rectangular waveguide, that is, making 


Ty2 = 0. ( }] ) 
The dash over the characters is to distinguish them from those of the 


branching cavity: 





pti 1Ag 
7 rs a ] -+- cos A) 
7 1Ad 
| Tas (1 + cos &) 
___—itane 
sis (T35)°(1 + eos 6) 





= 1(26 ) 
Ses = et uv 


At midband, Ag = 0 and 


Ses = 
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This means that at resonance the cavity acts as a short circuit located 
at half the length of the cavity. Again, as in the case of the branching 
cavity, if 

0 = (2n+1)r (45) 


there is no resonance. 


VI. SCATTERING COEFFICIENTS OF THE CHANNEL-DROPPING FILTER 


Knowing the scattering matrices of the branching and rejecting cavi- 
ties, we will find the scattering elements of the circuit obtained by join- 
ing a branching cavity and a rejecting cavity with a piece of waveguide 
of electrical length y. From the block diagram representation of the 
branching filter in Fig. 12, 


B, = See 4. A Ses : ( 16 ) 
B, = So7( l + A - ( 17 ) 
B, =- Be af , (48) 


B 


= Seg + A Seg, (49) 


BSege ‘eal (50) 


First it will be demonstrated that, if certain conditions are satisfied, 
port | of the filter (Fig. 12) is matched at all frequencies; then the val- 
ues of Bs and B; will be ascertained. 

From (46), (49) and (50) 


{ (s ¢ See g 2 i2y 
- 6666 — ZW 68 € a 
66 (51) 


’ <I t2y 
lL — SoeSeee 





B, — See 


Replacing Ses , Ses and Sep in the numerator with the values given in 
(36), (38) and (42), one obtains 


M35 |? (1 + cos 4) 
iAg 
I'35 at + cos 0) 


Yo i2y 
1 — SeeSese ” 





(52) 
i: ae. nes 
B, = See 





In order to have port | of the channel-dropping filter matched at all 
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al 


Bs 
Fig. 12 — Block diagram of the channel-dropping filter. 


frequencies, the reflected wave B, must vanish. The conditions to be 
satisfied are 


(53) 


= O34 ? 
Ag, 
v 


9 


634 + O05 — py — 


T 
7 — >= —5 (1 + 2p), 


where p is an arbitrary integer, and 
| Ti |? = 4| Tas |7(1 + cos @). (58) 

Conditions (53), (54), (55) and (56) state that, except for a small- 
length correction due to the effect of the coupling to the dropping wave- 
guide, both resonating cavities must be equal. Condition (57) estab- 
lishes that the distance between the centers of the resonating cavities 
must be an odd number of quarters of guided wavelength of the non- 
resonant mode. Since this condition is fulfilled rigorously only at dis- 
crete frequencies, the length of the cavities and the distance between 
them must be selected as short as possible. Finally, condition (58) states 
that in the branching cavity, Fig. 7, the power coupled from the reso- 
nating mode TEK,:° to the dropped mode TE,»", must be equal to four 
times the power coupled to each one of the TEq° ports. 

Substituting (53) through (58) in the expressions of the branched 
and through waves (47) and (48), one finds that 
. 1 
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The loaded Q is, as expected, inversely proportional to the power cou- 
pled into the resonant mode, but that coupling is not enough to insure 
a finite Q,. If 4, the electrical path difference discussed above and 
shown in Figs. 3 and 7, is an odd multiple of 2, then Q, becomes infi- 
nitely large. Also, as expected, at the frequency at which the resonating 
mode passes through cutoff, \,2 becomes infinite and Q, diverges. 

It can be shown that (68) and (69) are the transfer coefficients of 
the low-frequency circuit in Fig. 2, and consequently this circuit is the 
equivalent to that in Fig. 1. 

For the purpose of testing the cavities independently of each other 
it is important to know their scattering coefficients. They are obtained 
by substituting (66) and (70) in (36), (37), (38), (39), (42) and (43): 


l 


a] 1(2034—Y1) — 
See = <—¢ , 





Ses = e! (7834 ¥1) ig : a 
1 
fo 


I 


7 


Considering first the branching cavity, from (71) to (74), it is con- 
cluded that at midband the amplitude of the reflection at any port is 
one-third and the amplitude of the transmission to any other port is 
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At resonance, Ay = 0 and B, and B; become 


/ 9 12) 


(¢9/2+0 by /2+6,,/2 
B, - ere 347-¥1 11 ; 


B; = 0. 


Far from resonance, 


“% (63) 
2/Tss |? (1 + cos 6) 


(64) 

(65) 

In order to introduce the concept of loaded Q or Q: of the channel- 
dropping filter, the value of Ag will be expressed as a function of fre- 
quency. Ag is the difference between the electrical length of the resonat- 
ing cavity at midband fy and at any other frequency fo + Af. From (15), 
l 
fi 


to aw 4 20 wd + Af - 
Ao- "Oe 


a= A — 
ae f dfo _ To 


( Ao0 + 455 ) (66) 


(*:) > (67) 
> — : 4 
Age Jo 


provided that 


in which Ap and A,e are the free-space wavelength and guided wavelength 
of the resonating mode at midband fy , and Woo is the midband electrical 
distance between the diameter discontinuities of the branching cavity 
measured in terms of the resonating TE 2° mode. 

Substituting (66) in (59) and (60) leads to 


l 


+034 —¥1/24011/2 
Af’ (68) 
1 + 22Q, f 
Jo 
, Af 
12Q). ;. 


Af 
+- 120); p 
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The loaded Q is, as expected, inversely proportional to the power cou- 
pled into the resonant mode, but that coupling is not enough to insure 
a finite Q,. If 0, the electrical path difference discussed above and 
shown in Figs. 3 and 7, is an odd multiple of 7, then Q, becomes infi- 
nitely large. Also, as expected, at the frequency at which the resonating 
mode passes through cutoff, \,2 becomes infinite and Q, diverges. 

It can be shown that (68) and (69) are the transfer coefficients of 
the low-frequency circuit in Fig. 2, and consequently this circuit is the 
equivalent to that in Fig. 1. 

For the purpose of testing the cavities independently of each other 
it is important to know their scattering coefficients. They are obtained 
by substituting (66) and (70) in (36), (37), (388), (89), (42) and (43): 


oe, 
Seo = a ities hae ae i 
oe : 1) 
3 4 44 = Q, ; 





ej 

a4 f 
e' (7034 Wit. ae : 
1 + 24 a Q1 


Considering first the branching cavity, from (71) to (74), it is con- 
cluded that at midband the amplitude of the reflection at any port is 
one-third and the amplitude of the transmission to any other port is 
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two-thirds. Furthermore, the half power band of the reflection charac- 
teristic, (71), is 


3 fo 

2Q.° 
For the rejecting cavity, the loaded Q, (75) or (76), is twice the loaded 
@ of design of the channel-dropping filter. 


VII. RESUME OF FORMULAS FOR THE DIMENSIONING OF A CHANNEL- 
DROPPING FILTER 


The information given is the midband frequency fy and the half power 
bandwidth of the dropped channel 2Af, , defined in terms of the loaded 


Q): 


_ So 
Daf 


Qi 


The unknowns are: 

Yo, midband electrical distance between diameter discontinuities of 
the branching cavity measured in terms of the resonating mode, 

Yo, midband electrical distance between diameter discontinuities of 
the rejecting cavity measured in terms of the resonating mode, 

va, midband electrical distance between centers of cavities in terms 
of the nonresonating mode, 

2T35(1 + cos 6))3, coupling coefficient between the resonating mode 
and the through waveguide, 

/20 2, coupling coefficient between the resonating mode and the 
dropping mode. 

From (15) and (22), 


V0 = Boo + O55 ad [35 ‘i sin A -+ 2s 


where 6) reproduced from (14) is 


A -Wio + O34 + 835 — O45 (79) 


and Yip is the midband electrical distance between diameter discontinul- 
ties of the branching cavity measured in terms of the nonresonating 
mode; @,,, is the phase of the scattering coefficients between ports m 
and n, Fig. 4, with waveguide lengths reduced to zero; l, is the length 
of the branching cavity and \,» is the midband resonant mode guided 
wavelength. 
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lor the rejecting cavity 02. = 0; then, from (78), 


5 2 2a, 


Yoo -_ O55 — | T'35 q sin A + 27s =- P 
Ne 


ly, being the length of the rejecting cavity, and 
A, _ —Yro + O34 + O35, ros 45, . 
From (57), 


Vi Pro ‘ us ‘ ) 
ey — 2654 = = UE - Sp) (82) 


Wa _ Wo + 


where Y is the midband electrical distance between cavities and p is an 
arbitrary integer. From (58) and (70), 


Ag? , d 
Yo Xo = fo df (022 + 455) 
40), 





| I'35 * ] + cos A) ss: 


; aie 10 
From the theory of diffraction by small holes, 


_ 4a /y| He 


€ Ps o 


re wm Hy | | He 


(84) 


* (85) 


) 
3 Xo i] «e ~V/P,P, 


where c is the radius of the round hole that couples the resonating to 
the branching mode, and yu and «¢ are the permeability and permittivity 
of free space. If one considers that the standing resonating field is made 
of two waves propagating in opposite directions, | H2| is the absolute 
value of the magnetic field of one of those waves at the hole and P, is 
the average power carried by such a wave; | H,| is the absolute value 
of the magnetic field at the hole of a wave in the branching waveguide 
and P, is the average power carried by such a wave. 

The values of 022, 431, 055, 9 , | U'ss |, 41, He, Pi, P2 depend on the 
particular structure selected for the filter. 


7.1 Channel-Dropping Filter from Mode TEy,° in Circular Waveguide to 
TE yo in Rectangular Waveguide 


In Fig. 1 let us call a and 6 the radii of the double- and single-mode 
regions respectively, and W and d the width and height of the rectangu- 
lar waveguide. 
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From Ref. 1 


2a 


J y[v2/ es" 6 )| 


4| Yo | i(1 —- a ead Ue ) 


(87) 


(88) 


Jo(ve) Jilui(1 — 6)] 7 
vy? Jo( v1) Ji{v21 — 5)] 


where 


£4/) 
WY 


i 
oe re (92) 
a 


J, is the Bessel function of the first kind and order n, v, is the pth 


root of the J; function. 
From Ref. 12, pp. 58-59, 


2 ‘€ 
St ele (93) 


Ps Ao 
ant 4/1 - (Bey * 


From Ref. 12, p. 55, if the round hole is at the center of the rectangu- 


lar waveguide cross section, 
2/5 1 (3 ‘) , 


a Wd 








ae 
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Substituting (93) and (94) in (84) and (85) results in 


9 2 3 
a2” Aol 
Bo2 aa 9 


VoA a‘’ 

a ~- (ae 
r NV 1 1 - (Bs) 

_ (25) ] . 
Sue 2wW c 

—_—_—_—_—_—-— ~-- ; (96) 
3m! i — vedo 2 | (Wd)ia? 

27a 


Expressions (78), (79), (80), (81), (82), (83), (86), (89), (95) and 
(96) are the general relations necessary to determine the dimensions of 
the filter. As an aid in their solution it is convenient to have the approxi- 
mate results obtained when the expressions are drastically simplified by 
the following assumptions: all corrective terms due to coupling effects 
are neglected; the cutoff radius for TE .° at midband is selected 
a(l — 6/2); and 





| Ti | - 


These approximate results are 


aly V5 2alevV i 


Yo = P20 = 055 + 248 = i i (98) 


2nl» 


0 = 0 = —dw tr = ns (99) 


va = 0 + vo = * (1 + 2p), (100) 


35 3 l + cos = ) = Me j = ¥20 7 (101) 
2/5 { 1015 


O55 = ; 


(103) 


. : (104) 
(Wd)ia? 
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1.117 Mo (1 +4), (105) 
1.117 me ~ 5)» (106) 


(107) 


and explicitly, 


1 + 4s 
to), (108) 
V6 


Qu” Wd(1 + 4s) 


= No < 327 . y 
Ao Q.6 l - oW 


Wd(1 + 4s) 


27). 3! VT? > 
Ao Q16 E _ (2*,) | 
6 (1 — ¢os 7%) = (*) E ‘~ “| (‘ 4. =) 
7 167 Lvs Qi 








}14 


4 0.639 (1 a ay. 
1 


Since Q, and \» are given, the dimensions of the filter can be obtained 
by calculating 6 from (110) by successive approximations and substitut- 
ing this value in the relations (105) through (108). 

Far from resonance, the amplitude of the reflection of the Tly.° mode 
at each diameter change, derived from Ref. 11, is 


til = |Tw| = ") > _ 0.158. (111) 


Vo z 


7.2 Channel-Dropping Filter from Mode TEw° in Rectangular Wave- 
guide to TE" in Rectangular Waveguide 


Calling a and b the widths of the through waveguide in the double 
and single mode regions, respectively, W the width of the branching 
rectangular waveguide and d the height of all of them, one obtains from 
Ref. 11 
Y., (sin 296)’ 


65, = —2 arctg — — 
, B Ys (276 ) 
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| Voy | 
035 — O45 = 2 ; + 7, 
} 1b 


O34 -_ 0, 


i" Y 12 (88 sin 75 \° i+ > 
Tal" = 945 sin oes) Te 
2a \o $ - we ae 


where 


€ \ (ey on 
/ — 5 ) 

; Me 2b 
€ nro 3 = 
2W1~ (Gr) aan 
(118) 


From Ref. 9, p. 55, if the round hole is at the center of the branching 
rectangular waveguide cross section, 


- 
H? é. fee 





P, o\? aid’ 
V1-(3) 


€ do Ff 
i. =. s oo (sir) (120) 
VP; Wa 





Substituting (119) and (120) in (84) and (85) yields 


l6r Noe 


) aid 
_ Xo “Ty 
Qe 2W c 


3 (* = | (ae W)) 


a 





O29 = 


—- 
| Tie 


Again, expressions (78), (79), (80), (81), (82), (83), (86), (89), (95) 
and (96) can be simplified under the following assumptions: All correc- 
tive terms due to coupling effects are neglected; the cutoff width for 
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TE" at midband is a(1 — 6/2); and 
276 K 1. 
Then 
2alaV/ 5 - Qalpv/5 


Ao do 


—Yi tr= —~— 47, 


2/5 


T 
( 2p) 
= {1 + Bp), 


~_ 


Yor | 
40) 15 © 


(129) 


(130) 
(a' “4 sd’ 


and explicitly 


s) 


_ Ao( 1 + 4s) 
4/5” 


M (1 + 2p — 1 + *), 


9 Wd'(1 + 4s) 
48 } 0 
"aat[i - (20) Pad] 


=" 10.358 _— Wad'(1 + 4s) 


aut [1 — (9) J 


Qr(1 + 4s) 
64Q, 


Xo 
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Given Q, and Xo, the dimensioning of the filter is obtained by calculating 
5 from (136) and replacing this value in the explicit dimensions (131) 
through (135). 

Far from resonance, the amplitude of the reflection of the TE,w? mode 
at each width change, derived from Ref. 11, is 


| Tn | = | Tis | = 0.1256. 


VIII. DESIGN OF MODE-CONVERSION BAND-REJECTION FILTERS 


In order to build multipole mode-conversion band-rejection filters, it 
is necessary to know explicitly the scattering coefficients of a single cav- 
ity. These coefficients, given in (42) and (43), can be rewritten with 
the help of (66) as 

<] (2 ) 
See = =, O34-V1 


1+ 120, 
0 


Af 


1201 
i. — pg (2%0s4—-¥1) ' 


1+ 20, 


’ 


where 





bi) ~ 
do Th 

| Ts “(1 + cos 6) 
Comparing this equation with (70), we conclude that the band rejected 
by the band-rejection filter has half the width of the band dropped by 
a channel-dropping filter using the same rejection cavity. This coincides 
with the final remark of Section VI. The formulas that yield the dimen- 
sions of the rejecting cavity in Section VII can be used, replacing Q, 


by Q:, Pa 


IX. INTRINSIC Q OF MODE-CONVERSION BAND-REJECTION FILTERS 


By definition, the intrinsic Q of a resonating cavity is 


energy stored 
@ = of 
power dissipated as heat 
where w is the angular frequency. 
Let E be the electric field of the resonating mode at any point at the 
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instant when it passes through a maximum, H the magnetic field at the 
metallic boundary at the instant when it passes through a maximum, 
a the conductivity of the metallic wall, — the skin depth and yu and e 
the permeability and permittivity of free space; then 
E*dv 
Q = wote ———_, 


| H'ds 


where v and s are the volume and surface of the waveguide. 
For the case of the circular electric filter, the fields inside the cavity 


: 9 
9 7 av 
Ein = Jil ve cos z (142) 
1 1 ; 
a Age 


ly 2 
Hin -= ae vo) COS — 2 (143) 


wy Ag2 


are 


Outside the cavity, because of the boundary conditions of continuity of 


the tangential field components, they are 


. l 9 , ‘ 
Meat ee on (- ) cos _ ia : (144) 


Ve wl _2e/doo e)—1/: " 
Hou = — Jove) "iis oa (145) 
buy g2 
The axial coordinate z has its origin in the center of the cavity; the 
length of the cavity is /; and 





No 
Agtiout . f 


voAo ~ | R ( 146) 
4 Qnb} 


Substituting (142) through (145) in (141) leads to 


b 
° r 
: | Ja (. r) rdr 


Qre,.° = woke 
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cea 13 
since 


b r 2 
Ps P Ue - ° J ; Vo ), 


(150) 


(147) becomes 


where £, the skin depth, is 


2 s 
g&= 4/—. (152) 
4 WUC 
The result (151) coincides with the intrinsic Q of an infinitely long cy- 
lindrical cavity resonating with TE 92° at cutoff (Ref. 12, p. 59). Similar 
reasoning for a mode-conversion band-rejection filter in rectangular 
waveguide yields for the intrinsic Q of the resonating TE." mode: 


d 


Cre», > 9 ’ 
é(1 + =) 
a 


where a and d are the width and height of the rectangular waveguide. 
Typical theoretical values in copper waveguides are the following: 
For TE 2° mode at 5.4 millimeters, a & vo(5.4/27) = 6.04 millime- 
ters, and 


Qre,.° = 21,400. (154) 


This theoretical intrinsic Q is very large compared to the intrinsic Q 
obtainable in a parallelepiped-shaped cavity. For comparison we caleu- 
late the intrinsic Q of a half-wavelength cavity at 5.4 millimeters in the 
standard RG98U waveguide (0.074 X 0.0148 inches) that we assume 
to be made out of copper. Using the expression for intrinsic Q given on 
Ref. 12, p. 55, 


7 3460. (155) 





176 THE BELL SYSTEM TECHNICAL JOURNAL, JANUARY 1961 
For TEx»? mode, assuming a & Ay = 1.2 inches and d = 0.4 inches, 
expression (153) yields 

Qreec = 9000. (156) 


The intrinsic Q of a half-wavelength resonator in RG52U waveguide 
(0.4 X 0.9 inch) that we assume made of copper is 


Qre,.0 = 7990. (157) 


X. EXPERIMENTAL RESULTS FOR CHANNEL-DROPPING FILTER FROM T° 
TO TEi0 


We shall go first through the detailed design procedure of a channel- 
dropping filter for which the bandwidth is relatively large in order to 
show the limiting possibilities of these mode conversion filters; the ex- 
perimental results are quoted later. 

The selected center frequency and bandwidth of the dropped channel 
are 55.5 kme, (Ap = 5.4 millimeters) and 500 mc. The loaded Q is there- 
fore 


Q, = 110. (158) 


To dimension the filter roughly, we use (105) through (110). We shall 
use primes to distinguish the approximate sizes from those that are final. 
From (110), adopting s = 0, 


i >0.1 
and from (105) through (108), adopting p = 3, we find 
a’ 0.249 inch, 
b’ = 0.226 inch, 


ly = lp = 0.168 inch, 


iy 0.204 inch. 


If the branching rectangular waveguide is RG98U, W = 0.148 inch, 
d = 0.074 inch and the diameter of the coupling hole to the branching 
arm results, from (109) 


2c = 0.105 inch. 


Since this value is bigger than the 0.074-inch height of the rectangular 
guide, a round coupling hole can not provide enough coupling. There 
are many ways to increase the coupling. One is to build the rectangular 
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waveguide with its axis not perpendicular to the axis of the round 
waveguide but parallel to it, providing the coupling through a hole in 
the narrow wall. One of the ends of the rectangular waveguide must be 
short-circuited at an odd number of quarters of guide wavelength from 
the center of the coupling hole. For a fixed size of the hole, the amount 
of coupling can be increased by decreasing the width of the waveguide 
W, because the waveguide gets closer to cutoff. Another solution con- 
sists in wrapping around the TE,.° cavity a rectangular waveguide and 
providing the necessary mode selective coupling between them by means 
of several holes. The details are given in Ref. 2. 

The obvious third solution, and the one we adopt, is to increase the 
size of the coupling hole to the total cross section of the rectangular 
waveguide. If the coupling is too large, it can be decreased by displac- 
ing the hole to one side of the cavity. 

The strong perturbation of the field in the branching cavity due to 
such a large coupling hole modifies the scattering coefficients calculated 
in previous paragraphs, and the final length of this cavity, as well as 
the distance to the rejecting one, must be selected experimentally. The 
discrepancy between theoretical and experimental values is not large. 


10.1 Design of the Rejecting Cavity 


The design of the rejecting cavity requires the simultaneous solution 
of (80) and (83) for the determination of the three quantities a, b and 
lr. Thus, one of those quantities can be selected arbitrarily. 

A good criterion for this selection consists in demanding that at mid- 
band frequency the cutoff radius for TE 2° is 


vero _ Sa = @ (1 ~ 3) (159) 


- - 


because with this selection midband is equally separated from the two 
extreme frequencies that limit the proper operation of the filter. These 
are a lower frequency that cuts off the TE,:° in the large waveguide, 
invalidating the inequality (67), and an upper one that cuts off TEo.° 
in the smaller waveguide and above which propagation of TE 2° in that 
waveguide starts. 

Incidentally, it is interesting to notice that, for the frequency f = 


fo + Af, 
, 2 f wha 2) ‘ , 
(4) =f: -|— [1 +) 
Age | 2ra (1 + ‘) | = 
t fo/ 1) 
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becomes, through the use of (159), 


e ) ii (3 rs “ 
doe fo 


and the inequality (67) can be written 
Af 
— « 6. (162) 
fo 

This implies that the approximations hold as long as the relative fre- 

quency departure from midband is small compared to the relative diam- 

eter change. 

Another criterion for the selection of a, 6 or lp may arise from the 


advantage of using standard-size waveguides already available, as long 


as the limiting frequencies discussed previously are not approached. 
Following this procedure for a standard laboratory waveguide with 


21 = 7% inch, (163) 

the simultaneous solution of (80) and (83) yields 
2b = 0.5 inch and (164) 
le = 0.234 inch. (165) 


The measured performance of this band-rejection filter is shown in Fig. 
13. The agreement between theoretical and experimental values is excel- 
lent. 


10.2 Design of the Branching Cavity 


Ignoring the effect of the coupling hole between the branching wave- 
guide and cavity, the dimensions should be those of the rejecting cavity 
given in (163), (164) and (165). 

The distance between centers of the resonating cavities, according to 
(82) with p = 4, should be 


la = 0.572 inch. 


The number of quarters of wavelength between centers of cavities is 
nine. Experimentally it was found impossible to reduce J; because the 
TE? mode, being close to cutoff in the small waveguide, couples to 
the other cavity. The final dimensions, as well as the performance of the 
assembled channel-dropping filter, are shown in Fig. 14. 

The relatively high insertion loss for the dropped channel cannot be 





MODE-CONVERSION FILTERS 





0.234” 
— 
| | 


°o 


TI, 


to) 





THEORETICAL | EXPERIMENTAL 
270 MIDBAND (55.5 KMC/SEC | 56.35 KMC/SEC 
MC/SEC 
LOADED Q 222 209 
INTRINSIC Q 21,400 11,800 


RETURN LOSS IN DECIBELS 
a 








i 








i?) 1 it it 1 
55,800 56,200 56,600 57,000 
FREQUENCY IN MEGACYCLES PER SECOND 


Fig. 13 — Performance of circular-electric band-rejection filter. 


accounted for by heat losses because of the good performance of the 
band-rejection filter. Thus, mode conversion due to the asymmetry of 
the coupling to the rectangular waveguide must be its cause. Loss should 
be reduced using distributed coupling to the rectangular waveguide, as 
in Ref. 2. 

Pictures of the filter are shown in Figs. 15 and 16. 
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Fig. 14 — Performance of mode-conversion channel-dropping filter. 
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Fig. 15 — Mode-conversion channel-dropping filter. 


10.3 Band Rejection Filters in Different Waveguides 


Figs. 17, 18 and 19 show the geometry and experimental results for 


different band-rejection filters in round and rectangular waveguide. 


Those filters that have constant metallic cross section have generation 
and resonance of a higher-order mode in the region where the dielectric 
is located. For the case of Fig. 19, it has been shown in Ref. 11 that a 
rectangular waveguide with a dielectric slab is equivalent to a rectangu- 
lar waveguide that has a width increase for a length equal to that of the 
slab. Calling d the width of the slab, ¢ the distance to the near narrow 
wall and ez the permittivity of the dielectric, the relative apparent in- 


ara 


Fig. 16 — Exploded view of mode-conversion channel-dropping filter. 
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With this value known, all the design formulas in Section VII can be 
used. 

In the round waveguide in which only circular-electric modes are of 
interest, tuning is available by changing the physical length of the reso- 
nating cavity. For that purpose a telescopic type of junction is ideal, 
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since the cracks do not interrupt the conduction current and since it is 
very easy to manufacture. One tube of inner diameter 2a, inside of which 
two tubes of outer diameter 2a and inner diameter 2) can slide, will 
suffice. 

The tuning in the case of Fig. 17 can be achieved by trimming the 
dielectric. For the filters in rectangular waveguide, Figs. 18 and 19, one 
tuning screw at each one of the electric-field maxima of the resonating 
mode provide the tuning. 


XI. CONCLUSIONS 


Resonance of higher-order modes in waveguides has been advantage- 
ously used to make very simple band-rejection filters of unusually low 
loss. In particular, the filter operating with circular-electric modes has 
an intrinsic Q that is one order of magnitude better than the intrinsic 
Q of conventional (cavity or lumped) band-rejection filters operating 
at the same frequency. 

Mode-conversion band-rejection filters have been used as building 


blocks for the construction of channel-dropping filters that simultane- 
ously produce the band separation and the transfer from TEq° mode 
to TE\~ required in the long distance waveguide communication sys- 
tem.’ One model operating at 56.3 kme has a bandwidth of 490 me, and 


the insertion loss for the dropped channel is | db. 

Although the emphasis in this paper has been on filters operating 
mainly with circular-electric modes in round waveguides and TE modes 
in rectangular waveguides, the calculations are quite general and can 
be applied in any scheme in which mode-conversion filters are used. 
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XIII. LIST OF SYMBOLS 


a = radius of the resonant cylindrical cavity or width of the 
resonant rectangular waveguide. 
radius of the through cylindrical waveguide or width of the 
through rectangular waveguide. 
radius of the coupling hole between resonant cavity and 
branching arm. 
height of any rectangular waveguide. 
frequency. 
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midband frequency. 

distance between cavities. 

length of branching cavity. 

distance between centers of cavities. 

length of rejecting cavity. 

number of nonresonant mode quarter-wavelengths between 
centers of branching and rejecting cavities (p is an arbitrary 
integer ). 

number of resonant mode half wavelengths in each cavity 
(s is an arbitrary integer). 

intrinsic Q. 

loaded Q. 

scattering coefficient of a half cavity or a more complicated 
circuit. 

width of the branching rectangular waveguide. 

admittance. 

scattering coefficient of elementary structures. 

relative diameter change or width change of through wave- 
guide. 

permittivity of free space. 

permittivity of dielectric. 

electrical difference between two energy paths. 


midband electrical difference between two energy paths. 


phase of the scattering coefficients of junctions with wave 
guides reduced to zero length. 

midband free-space wavelength. 

midband guided wavelength of the nonresonant mode. 
midband guided wavelength of the resonant mode. 
permeability. 

nth root of the J; function. 

conductivity of metal. 

midband electrical length of the resonating cavity in terms 
of the resonating mode. 

midband electrical distance between cavities in terms of the 
through mode. 

midband electrical distance between centers of cavities in 
terms of the through mode. 

electrical distance between the branching cavity discontinui- 
ties in terms of the non-resonating mode. 

midband electrical distance between the branching cavity 
discontinuities in terms of the non-resonating mode. 
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electrical distance between the branching cavity discontinui- 
ties in terms of the resonating mode. 
midband electrical distance between the branching cavity 


discontinuities in terms of the resonating mode. 


electrical distance between the rejecting cavity discontinui- 
ties in terms of the nonresonating mode. 

midband electrical distance between the rejecting cavity dis- 
continuities in terms of the nonresonating mode. 

electrical distance between the rejecting cavity discontinui- 
ties in terms of the resonating mode. 

midband electrical distance between the rejecting cavity dis- 
continuities in terms of the resonating mode. 
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A Cireular-Electric Hybrid Junction and 


Some Channel-Dropping Filters 


By E. A. MARCATILI 


(Manuscript received July 11, 1960) 


A TE ,° hybrid junction that operates similarly to the Riblet short-slot 
hybrid is described, but because the modes involved are circular-electric, the 
hybrid can be telescopically mounted, allowing for adjustment to almost 
any power division. The experimental results show that, centered at 55.6 
kme, the frequency range is larger than 20 per cent. Adjusted for equal 
power division, the balance is better than 0.5 db and the unwanted reflec- 
tions in the driven and balanced (isolation) arms are at least 23 db below 
the input signal. 

Using the hybrid together with band-reflection, band-transmission or 
high-pass filters, it is possible to build low-loss channel-dropping filters. 
In particular, the use of simple cutoff waveguides permits the design of 
filters with almost rectangular transfer characteristics. 


I. INTRODUCTION 


The importance of hybrid junctions for many purposes — measuring, 
filtering, balancing, equalizing, etc. — need hardly be emphasized. The 
long distance waveguide communication system! operating with the 
low-loss circular-electric TE ;° mode has only two hybrids available: 
the directional coupler, which has a fixed power division, and the optical 
hybrid,? which requires multimode waveguides. This paper describes a 
third hybrid, which operates like Riblet’s coupler* and which adds to 
the well-known advantages of that coupler the unique property of ad- 
justable power division. 

Adjusted for 3-db power division, the hybrid, together with mode- 
conversion band-rejection filters, band transmission filters or cutoff 
waveguides,” can be used as low-loss components of constant-resistance 
channel-dropping filters.’ The scheme that uses high-pass filters (cut- 
off waveguides) deserves special attention because the amplitude trans- 
fer characteristic of the dropped channel can be made to approach a 
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PORT b, TES, 


Fig. 1 Circular-electric hybrid. 


rectangular shape of arbitrary bandwidth. This permits not only re- 
laxing the demands on the guard bands between neighboring channels, 
but also the multiplexing of bands too broad (extremely short pulses) *® 
to be handled by mode-conversion filters. 


Il. DESCRIPTION OF THE HYBRID 


The hybrid consists of two coaxial circular metallic tubes, of which 
the inner one has a gap /, as shown in Fig. 1. The ratio of diameters 
selected is equal to the ratio of the second to the first roots of the Bessel 
function J; : 


1) 


The outer diameter D is chosen so that it cuts off the TEo3° mode at 
the highest frequency of design of the hybrid. 

The hybrid is made of two four-port junctions like the one of Fig. 2. 
It will be shown that power entering in any port is almost equally di- 
vided between the two forward modes. Consequently, going back to 


a - PORT 3, TEG) 
* 
\ 


PORT 4, TEG2 


PORT 2, TES, 


Fig. 2 Four-port junction. 
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Fig. 1, power entering in any port is almost equally divided between 
TEo:° and TEp2° in the gap region. Each one of these modes repeats the 
power division at the end of the gap, so the power collected in each 
output depends on the relative phases of the modes at the end of the 
gap. Since the velocities of these modes are different, the relative phase, 
and consequently the power division, can be selected arbitrarily by 
changing the length of the gap. 


III. PROPERTIES OF THE FOUR-PORT JUNCTION AND THE HYBRID 


The most general scattering matrix for the reciprocal four-port device 


of Fig. 2 is 
Su 2 3 Siu! 
Sie 
Sis 23 33 Ssa 
Siu ‘9 : Sa 


Entering port 4 with mode TE .°, since the metallic inner tube has 
its surface where the electric field is zero (first zero of the J; function), 
the boundary conditions are automatically satisfied, the TEo2° mode is 
unperturbed, and consequently the back-scattering 


(3) 


Furthermore, the forward-scattering coefficients at the plane where 
the coaxial waveguide starts are 


»3.832 


Ji (a)ada 
’ “0 
Su .7.016 
Ji (a)ada 
1 Jo 


| Jet a)ada 


“3.832 


So 4 


97.016 


Ji (a)ada 
{ 
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Assuming the junction to be nondissipative, (2) must satisfy, because 
of conservation of energy, the following unitary relations’ 


4 *e 

: l tl m=n 
Z SemSan* — ‘ef 
gal 0 if msn 


in which the asterisk means “complex conjugate of.”’ 
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From (3) and (6), 


Sie 


Ses 


where 6; is the phase of Sj; . 
Since S,, and Se, are known from (4) and (5), the five previous ex- 
pressions become 


1.078 Su, (12) 
1.163 Su, (13) 
0 0 + Jo\4 i613 P 
0.68(1 — 4.68 | Sy, |*)’e""?, (14) 


0.733(1 — 4.68 | S,, |2)4ei" (15) 


’ 


~ ’ 2613 ‘10 
—2.163 Sy*e". (16) 


In the experimental hybrid to be described later on, the modulus of 
the reflection coefficient is 


Su] < 0.05 
and consequently powers of S,, bigger than one can be neglected. With 


this simplification, the forward transfer elements of the scattering ma- 
trix of the hybrid (Fig. 1) are 


' 19 
dees Sis! a 
. 4 2 i2el/Xoo S13] 120,34 
Sac = Ou € . [ + |—| e 
S14 | 
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’ ’ t2rl/Xoo 120;2+i29r1/! 
= Sue o2(] = ¢ ts 


where 





CIRCULAR-ELECTRIC HYBRID JUNCTION 


is the beating wavelength between TE 9,° and TE° in the gap; 


Z A 
1 _ (3882\° 
/ rD 


r 


| _ (7010 
V rD 


are the TE9,° and TE.2° guided wavelengths; and \ is the free-space 
wavelength. 

For a given gap l, the power division of the hybrid K, and the phase 
shift between the two outputs are derived from (4), (5), (14), (15), (17) 
and (18): 





Noi 





hoo = (21) 


2 1.011 + cos (2 +: 


: 
Mac 
| 





Sas | 2nl 
1 — cos (20. — - =) 


Bac — Ow = &a t+ tg 13.15 VK — 0.0055. 


The possible range of power division K obtained from (22) is 
0.0055 =< K < om. (24) 


For K = 0.0055, the power flowing in the inner guide is a minimum 
and specifically 26 db below the input. For K = , the power flowing 
in the coaxial guide is zero. 

Since the beating wavelength A, as well as the argument 6); , are fre- 
quency-sensitive, the power division K given in (22) also varies with 
frequency. We have not calculated 6; , but it is known’ that the fre- 
quency dependence of 6;; and of A tend to cancel each other’s effect, 
allowing the power division K to be constant over a relatively broad 
band. Furthermore, it is very easy to adjust experimentally the gap / 
for any allowable power division K because the hybrid can be built with 
sliding tubes. The modes involved are circular electric and consequently 
the cracks do not interrupt conduction current lines. 


IV. EXPERIMENTAL RESULTS 


In order to make available the power from the hybrid a four-port 
transducer has been electroformed capable of transferring TE 9,° to 
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Fig. 3 (a) Cireular-electric hybrid assembled with TE2® to TE, 9 trans 
ducers; (b) exploded view. 


TEo° and TE9,° to TE (Fig. 3). The last change of modes is obtained 
by smoothly deforming a rectangular waveguide into a coaxial wave- 


guide. The transducer generates small amounts of unwanted higher 
order modes, which can resonate® and ruin the behavior of the hybrid. 


The resonances can be damped by using for the external tube of the 
hybrid a lossy-jacket helix waveguide,’ which substantially attenuates 
any mode with axial conduction currents. 

Fig. 4 shows the electrical behavior of the hybrid adjusted for equal 
power division. From 50 to 61.2 kme the balance is better than 0.5 db 
and the isolation better than 23 db. 

At 55.6 kme the power lost in the hybrid and transducer is 0.83 db. 
In order to prove that most of this loss occurs in the TE9,° to TE, 
transducer, the gap was enlarged until, at 55.5 kme, most of the power 
was recovered in the inner waveguide (K = «;1 = 0.906 inch). The 
measured insertion loss was then reduced to 0.3 db. 
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Fig. 4 — Performance of circular-electric hybrid and TEo.2° to TE,o9 transducers. 
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No efforts have been made to improve either the hybrid or the trans- 
ducers. The possible changes for the hybrid are of an experimental nature 
and consist in varying the diameter of the gap region and including cir- 
cular symmetric lumped discontinuities to improve the balance and 
decrease the unwanted reflections. The possible improvement of the 
transducer consists in passing from the relatively simple-to-build linear 
taper used in these experiments to more sophisticated designs'® that 
reduce mode conversion. 


V. CONSTANT-RESISTANCE CHANNEL-DROPPING FILTERS 


It is known that a constant-resistance channel-dropping filter (in- 
put matched at all frequencies) can be made using two hybrids connected 
by two filtering paths. The hybrid described in Section III lends itself 
to use with filters that operate with low-loss circular-electric modes, and 
is consequently attractive for use in the long distance waveguide com- 
munication system. 

The filters that most naturally suit the hybrid are those that possess 
circular symmetry. For example, filters made with inductive irises, 
mode-conversion filters‘ and cutoff filters. In Fig. 5, two such filters 
with identical transfer characteristics are located symbolically in the 
inner and outer waveguides connecting two circular-electric hybrids. 
TEo:° power that enters port 1, and is rejected by the filters, recombines 


as TEo,° in port 2. The power transmitted through the filters can be 


made to recombine either in port 3 or in port 4. On one hand, as- 
suming the gaps of both hybrids to be identical, power recombines in 
port 3 if the inner and outer electrical paths between planes a and b 
are identical, and power recombines in port 4 if those paths differ by x 
radians. On the other hand, assuming the two paths to be identical, 
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Channel-dropping filter using circular-electric hybrids 
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power recombines in port 3 if the hybrids are identical and recombines 
in port 4 if both gaps differ by half a beating wavelength between the 
TEo° and TEm° modes. 

Probably the most interesting of the channel-dropping filters is ob- 
tained by using cutoff waveguides (high-pass filters) in the connecting 
paths. The interest comes from the fact that the transfer characteristic 
of the dropped channel can be made to approximate a rectangular shape. 

Before considering the actual geometry of these filters we analyze 
the behavior of a chain of constant resistance filters represented sym- 
bolically in Fig. 6(a). The first link consists of two hybrids # connected 
by two paths of identical transfer and reflection coefficients. Each path 
includes a high pass filter that cuts off at frequency f, . The only differ- 
ence between the successive constant resistance filters is the cutoff 
frequency of the high-pass filters. Because of the phase-shifts between 
the different arms of the hybrids, and the similitude of the connecting 
paths, power entering in port 0 can be recovered only in ports 1, 2, 3 

(n + 1). The power transfers between input and output ports are 
given in Fig. 6(b); n — 1 channels can be dropped out of n constant 
resistance filters. 

The actual geometry of one of the units of the chain is very simple 
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Fig. 6 — (a) Chain of constant-resistance filters; (b) power transfer between 
ports 0 and 1, 2,3, --- , (n+ 1). 
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Fig. 7 — Constant-resistance filter. 


when circular-electric hybrids and cutoff waveguides are used as shown 
in Fig. 7. The ports 0, 1, Pp and Q correspond to those of the first unit 
in Fig. 6(a). The two hybrids in Fig. 7 are different, in order to recombine 
the power transmitted through the cutoff sections in the inner wave- 
guide. 

Without cavities we have achieved an almost rectangular transfer 
characteristic of arbitrary width. The guard band between successive 
channels can be made, at least in principle, arbitrarily small. A working 
model of cutoff filters has been demonstrated in Ref. 2. 

There is another channel-dropping filter worth considering because of 


its simplicity and because it uses the structure shown in Fig. 2 as a hy- 


brid. [The reader can check that the scattering coefficients of this june- 
tion given in (4), (5), (12), (13), (14), (15) and (16) are very close to 
those of a hybrid when Sj, is negligibly small.] 

Before considering the actual channel-dropping filter we shall de- 
scribe a microwave equivalent circuit, Fig. 8. It consists of two hybrids, 
indicated as Riblet couplers, which are connected by two waveguides 
of equal electrical lengths. These waveguides are also coupled through 
two identical resonating cavities. The electrical distance between cou- 
pling holes in the upper waveguide is an odd multiple of x and in the 
lower waveguide is an even multiple of z. 

Out of resonant frequency of the cavities power entering port 1 splits 
in equal parts in the first hybrid and recombines in port 5 of the second 
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Fig. 8 — Microwave equivalent circuit of channel-dropping filter of Fig. 9. 
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hybrid. At resonance, power entering port 1 splits in equal parts and 
each one excites the cavities in different ways. Let us follow the power 
in the upper path. Because of the distance between coupling holes, the 
cavities are excited in opposite phase and the reradiation from the cavi- 
ties is such that all the power flows back toward port 2 as if reflected 
from an equivalent short circuit located in plane of symmetry a. Mean- 
while, the power flowing in the lower path from port 3 excites the cavi- 
ties in phase and again, because of the adequate distance between holes, 
all the power goes back toward port 3 as if reflected by a short circuit 
in plane a. Recombination of the two waves reflected in plane a takes 
place in port 4 of the first hybrid. 

The actual microwave circuit for circular electric waves is shown in 
Fig. 9. The two hybrids are like those of Fig. 2. Waves flowing in ports 
2 and 3 of Fig. 8 are equivalent to the TEo,° and TE 2° waves in the gap 
of Fig. 9. The length of the gap region is one beating wavelength between 
the TEo:° and the TEm° modes; the diameter is selected in such a way 
that the TK;° is cut off except for two enlarged regions where resonance 
of this mode takes place.* These mode-conversion resonant “cavities” 
couple to both TE9;° and TE 2° modes and are separated by half a guided 
wavelength measured in TE 2° mode and one guided wavelength meas- 
ured in TE 9,° mode. The mode conversion “cavities” are therefore 
equivalent to the resonant cavities of Fig. 8. 

If the coupling between TE 3° and TE:° is different from the coupling 
between TE93° and TE°, the channel-dropping filter no longer has 
constant resistance. This can be deduced from Fig. 8 by making the 
coupling holes in the upper waveguide different from those in the lower 
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Fig. 9 Channel-dropping filter with TEo3° mode-conversion filter. 





CIRCULAR-ELECTRIC HYBRID JUNCTION 





Fig. 10 Rings to equalize coupling between TE 3° : TEo.° and between 
TEo3° and TE. 


To equalize the couplings in Fig. 9, rings like those shown in Fig. 10 
can be used. 

In all the filters described in this section, the dropped channel appears 
as TEo,° mode. It may be necessary to transduce this mode into TE,» 
There are essentially two techniques. One consists in using a broadband 
transducer like the one described in Fig. 3 of Section IV; another con- 
sists in using a transmission cavity that resonates with coaxial circular 
electric mode and that couples to the coaxial waveguide and to a ree- 
tangular waveguide." The second approach yields a much shorter trans- 
ducer but it is not broadband. 


VI. CONCLUSIONS 


A hybrid capable of dividing Tlo;° mode into TE9:° and TE 9;° has 
been described. It operates similarly to the Riblet short-slot hybrid, 


but because the modes involved are circular electric, the hybrid can be 
made of sliding coaxial tubes that allow adjustment to almost any power 
division. 

The experimental results show that, centered at 55.6 kme, the fre- 
quency range is larger than 20 per cent. Adjusted for 3 db division with 
the transducers from TEo,° to TE,” included, the balance is better than 
0.5 db and the unwanted reflections in the driven and balanced (isola- 
tion) arms are at least 23 db below the input signal. 

No efforts have been made to improve either the hybrid or the trans- 
ducers. The possible changes for the hybrid are of an experimental na- 
ture and consist in varying the diameter of the gap region and including 
circular symmetric lumped discontinuities to improve the balance and 
decrease the unwanted reflections. The possible improvement of the 
transducer consists in passing from the relatively simple-to-build linear 
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taper used in these experiments to more sophisticated designs” that 
reduce mode conversion. 

Using the hybrid together with band-reflection, band-transmission or 
cutoff waveguides, it is possible to build low-loss constant-resistance 
channel-dropping filters. In particular, the use of cutoff waveguides 
permits us to design filters with almost rectangular transfer character- 
istics. 

Hybrids and filters described in this paper operate with circular- 
electric modes, but their equivalents operating with TE modes in rec- 
tangular waveguides can be easily derived by the reader. The design 


of TE, mode conversion filters is given in an accompanying paper.‘ 
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Band-Splitting Filter 


By k. A. MARCATILI and D. L. BISBEE 


(Manuscript received July 11, 1960) 


A constant-resistance filter capable of dividing a very wide band into 
two subbands is described. It can handle one octave in the millimeter region 
with only 1.5 db insertion loss for each subband. The splitting transition 
takes place in a very narrow band (160 mc). Two of its components are 
important devices: an elbow and a hybrid junction. Both are quasi-optical 
and work with TE ° mode in 2 inch diameter waveguide. 


I. INTRODUCTION 


The long-distance waveguide communication system will handle an 
extremely broad band extending perhaps from 40 to 80 kme.’ For re- 
generation and amplification this band must be divided into channels 
around 400 me apart. Promising filters capable of performing this 
channel separation have been described elsewhere,“ but it is im- 
probable that satisfactory filtering can be obtained if approximately 100 
channel-dropping filters are to be stacked one after another. The main 
reasons for possible trouble are: 

(a) Resonance of unwanted modes. This occurs because some of the 
filters are required to operate over a range of frequencies covering more 
than one octave. 

(b) Multiple reflections. Although the reflection from each filter is 
small, the combined reflection of as many as 100 may become pro- 
hibitively large at discrete frequencies. 

Troubles from these sources can be reduced by dividing the broad 
40-kme band into several subbands. The width of the subbands can be 
adjusted to accommodate a suitable number of channel-dropping filters. 

This paper describes a filter capable of dividing a band in two parts. 
It can easily handle one octave in the millimeter region with low inser- 
tion loss because it operates with low loss mode TE :° mostly in 2 inch 
diameter waveguide. 

The splitting process can be repeated as many times as necessary by 
cascading similar filters. 


197 





198 THE BELL SYSTEM TECHNICAL JOURNAL, JANUARY 1961 


a L 
sccteed 








90°» ~0° “Zz 0° 
PORT | ——— H, _— PORT 2 PORT 3 ———={ Hz = PORT 4 
o° .) 90° oe “90° 














ee a 


Fig. 1 Band-splitting filter. 


A band-splitting filter has been built and tested. The results are quite 
promising. 


Il. BAND-SPLITTING FILTER 


The band-splitting filter is a constant-resistance filter, Fig. 1, made 
essentially of two identical hybrid junctions H; and He and two identical 
high-pass filters r. The phase-shifts between arms of the hybrids are 
shown in the figure. Power entering in port | is divided by the hybrid 
H, into two equal parts that travel through equal electrical paths toward 
the high pass filters. Frequencies above that of cutoff of the filters keep 
on traveling and recombine with the same phase in port 4, and the 
opposite phase in port 3. Frequencies below that of cutoff of the filters 
are rejected and add in port 2 and subtract in port 1. Consequently, all 
the power entering in port 1 is recovered in ports 2 and 4. 

What happens at frequencies where the hybrids are identical but do 
not divide power in equal parts? The unitary power entering in port 1 
is divided by the hybrid H, in two parts, T and 1 — I, that travel toward 
the filters. Due to conservation of energy the phase-shifts in the hybrids 
are independent of the value [’, and the powers appearing in ports 3 and 
$ (above cutoff) are 

P i— 47T(1 — Pf), (1) 
P, 1r(l — 7). (3 
Power recovered in the first hybrid (below cutoff) is 
P, L=— eri — Tf 
P» iT(l1 — T 
We check immediately that if the hybrids operate ideally splitting 
power in halves, Tl = 3 and 
P; = P3 = 0, 
P; P, = 1. 
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P, and Ps, given in (2) and (4) measure the recoverable power 
when the hybrids are not ideal. Let us plug some numbers into these 
expressions. For [ = 0.333 --- or T = 0.666---, Pe = Py = §. In 
words, even at frequencies where the power division of the hybrids is as 
bad as two to one, the recoverable power of the band-splitting filter is as 
high as eight-ninths of the input power (0.5 db loss). This good be- 
havior of the band-splitting filter, even with unequal power division in 
the hybrid, assures satisfactory operation over an extremely broad band. 

An experimental model of a band-splitting filter operating with 
circular-electric mode is shown in Fig. 2, and its schematic appears in 
lig. 3. It is interesting to note that we have used two elbows between 
the generator and the actual band-splitting filter just for ‘‘compact- 
ness.” 

We describe now the experimental technique used to evaluate the 
band-splitting filter and the results. 

Most of the experimental data is presented in oscillographs that carry 
frequencies in abscissas and power in ordinates. The insertion loss of a 


device, for example, can be calculated from two oscillographs which 


show the transmitted powers with and without the device included in 
the microwave circuitry. 

The wide band-sweep displayed in each oscillograph has been achieved 
through the use of backward wave oscillators, but we had to pay a price 
in that the output power of these tubes varies rapidly with frequency, 
and therefore the oscillographs exhibit a fine structure that makes 
measurement a little cumbersome. 


Fig. 2 — Band-splitting filter for TE’. 
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Fig. 3 Schematic of a band-splitting filter connected to a generator 


There are several sources of errors in our measurements. One stems 
from the assumption that we have a square-law detector; this is not 
strictly so, and consequently only the comparison of similar ordinates 


gives reliable quantitative results. Furthermore, small ordinates yield 


only qualitative data, because it is for small signals that the detector 
departs strongly from the square law. The other sources of errors are 
the multiple reflections of the through mode and the damped resonances 
of spurious modes. Both exist only because of the measuring technique, 
and consequently do not represent electrical properties of the devices 
under measurement. In effect, it is known that discontinuities in a 
multimode waveguide excite practically no reflections except forward 
conversion. Now, in order for measurements to be made, the multimode 
waveguide must be connected to a generator and receiver that operate 
in single mode rectangular waveguides. It is in the connecting transducers 
where most of the reflections of the through mode take place and also 
where the converted modes are cut off and reflected. 

Multiple reflections and damped resonances show their presence in 





BAND-SPLITTING FILTER 201 


the oscillographs as very fine periodic oscillations superimposed on the 
already jagged backward wave oscillator output [see, for example, Fig. 
8(d)]. In transmission, where is the correct reading, at the top of this 
fine structure, at the bottom, or someplace in between? On one hand, 
admitting that the fine structure is due exclusively to multiple reflec- 
tions between the input and output transducers, the correct reading is 
at the top because the discontinuities in the transducers act like irises 
located outside of the nonreflecting device being measured, and con- 
sequently these irises can only reduce the transmission, never increase 
it. On the other hand, admitting that the fine structure is due exclusively 
to resonance of spurious modes, the correct reading is half-way between 
the top and the bottom since the transmission can be increased or de- 
creased with resonances.’ A fair compromise between the two extreme 
readings is the average. 

Now we can look at the results. The outputs of the band-splitting 
filter of Fig. 3 are shown in Figs. 4(a), (b) and (c)}, together with the 
reference output of the generator, Fig. 4(d). 
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Fig. 4 — (a) Output of port 4; (b) output of port 2; (c) output of port 3 [taken 
with 6.0 db greater sensitivity than (a), (b) and (d)]; (d) reference output. 





202 THE BELL SYSTEM TECHNICAL JOURNAL, JANUARY 1961 


Far from cutoff the band-splitting filter has 1.5 + 0.1 db insertion 
loss for each subband [compare Figs. 4(a), (b) and (d)]; but even as 


close to cutoff as 75 me the insertion loss is increased by only | db. The 


band lost because of the splitting filter is very narrow. 

It will be shown in Sections III and V that, of the 1.5 + 0.1 db inser- 
tion loss for either subband, ~0.5 db is lost in each hybrid and ~0.2 db 
in each elbow. This accounts for ~1.2 db; the remaining 0.3 db must 
be attributed to losses in the rest of the circuitry. Any substantial re- 
duction of losses will have to come from improvements of the hybrids. 

How do we adjust the band-splitting filter? The requirement is that 
the power recombining at each hybrid must follow equal electrical 
paths. Frequencies below cutoff require path 1, in lig. 3 to be identical 
to L;’, and frequencies above cutoff must have L; + Le identical to 
Li + Le’. The fact that the adjustment for frequencies below cutoff 
is independent of Lz and Ly’ suggests a two-step procedure in which the 
second does not alter the first: 

(a) trim Lt; and 1,’ for minimum power in port | (maximum in 
port 2); 

(b) trim Le and Le’ for minimum power in port 3 (maximum in 
port 4). 

We describe next each one of the band-splitting filter components. 


lil. HYBRID JUNCTION 


Consider an infinite volume of metal in which two infinitely long 
cylindrical holes of equal diameter are bored in such a way that the 
axes are coplanar and normal to each other, as in lig. 5. We thus have 
two cylindrical waveguides making a cross. TE»:° mode fed in one of 
the arms passes straight through the junction almost unperturbed 
provided the diameter of the waveguide is much larger than the free- 
space wavelength. The reason is that we are dealing with an almost 
optical problem. 

Now let us include in the junction a thin plane sheet of a material 
to be described later. The sheet passes through the intersection of the 
axes of the waveguides and makes an angle of 45° with each of them. 
This thin layer acts as a semitransparent mirror, and TE ,° mode fed 
in one of the arms is partially transmitted straight through and partially 
reflected to one of the side arms. If the power division is half and half, 
the junction becomes a hybrid. If all the power is reflected (sheet of 
metal) the junction becomes an elbow, as in Fig. 6. 

The semitransparent mirror can be obtained with sheets of dielectric, 
wire mesh, evaporated film, ete. 
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hybrid in 2-inch waveguide; (b) exploded view. 
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elbow in 2-inch waveguide; (b) exploded view. 
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Part of the power sent through one of these hybrids is converted to 
unwanted modes, a good part of which could be recovered by modifying 
the mirror slightly. In effect, the incident mode can be considered as the 
superposition of an infinite number of plane wavelets all traveling almost 
parallel to the axis of the waveguide and each one impinging with dif- 
ferent polarization on the semitransparent mirror. As a consequence of 
the polarization each wavelet has its own reflection coefficient. Mode 
conversion can be avoided by making the local reflection coefficients 
identical; this can be achieved with a nonuniform semitransparent 
mirror — for example, if the sheet is dielectric or wire mesh, the thick- 
ness of the dielectric or the density of the holes must be a function of 
the azimuth. 

Let us see the experimental results. Two hybrids were assembled 
using glass for the semitransparent mirrors. The thicknesses were de- 
termined experimentally to provide 3 db power division in the 50 to 60 
kme band. Three sheets of glass with a total thickness of 0.018 inch 
were assembled into Hybrid No. 1. The other, Hybrid No. 2, was as- 
sembled with four sheets of glass with a total thickness of 0.021 inch. 
The power division did not change rapidly with thickness, and the two 
mirrors appeared to be the best combinations obtainable with the 
available glass sheets. 

Similar measurements were made to determine the performances of 
both hybrids and, since the results were very similar, Fig. 7 shows only 
those for Hybrid No. 1. The outputs from arms 3 and 4, Figs. 7(a) and 
7(b), are nearly equal, showing close to 3 db power division. 

A method to check the insertion loss and the balance of the hybrid 
is described next. The four port hybrid is reduced to a two port structure 
by placing reflecting pistons in arms 3 and 4, Fig. 7(c). By adjusting 
the relative position of the pistons, the power transmission is maximized. 
With this scheme the transmitted power crosses the hybrid twice, once 
going toward the pistons and second, bouncing from them. Figs. 7(c) 
and (d) show the transmitted and reflected levels. The reference level is 


shown in Fig. 7(e). The two-way loss of the hybrid is approximately 
1 db + 0.1 db. The reflected power from the hybrid with the pistons is 
very small [see Fig. 7(d)] because it is quite similar to the power re- 


flected in the internal mismatches of the measuring set, Fig. 7(f). This 
indicates first that the 1 db insertion loss is not due to reflections but 
rather to mode conversion; second, that over the range from 51.0 to 
60.8 kme the power division in the hybrid is frequency-insensitive, since 
otherwise the pistons could not tune out the reflections over this band. 

The responses of Hybrids Nos. 1 and 2 from 60 to 68.4 kme and from 
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Fig. 7 — Hybrid No. 1: (a) output of port 3; (b) output of port 4; (c) output 
of port 2 with pistons on ports 3 and 4; (d) output of port 1 with pistons on ports 


3 and 4; (e) reference output; (f) reflection from measuring set 
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65.9 to 76.4 kme are shown in Fig. 8. The outputs of arms 3 and 4, which 


should be equal in the ideal case of 3 db power division, have been super- 


posed for comparison. Figs. 8(a) and 8(b) show that those outputs are 
very similar in Hybrid No. 1, and consequently power division has little 
frequency dependence. On the other hand, Figs. 8(¢) and 8(d) indicate 
that power division in Hybrid No. 2 gets worse as the frequency in- 
creases. At the highest frequency, 76.4 kme, the output ratio is close to 
two to one. 

Why the dissimilar frequency behavior? The answer may be that, 
because of the difference in thickness of the semitransparent mirrors, the 
hybrids achieve the ideal 3 db power division at different frequencies, 
and consequently one of them is bound to behave better than the other 
in the range of our measurements. These hybrids are so broadband that 
when they were assembled to provide 3 db division in the range from 50 
to 60 kme they looked very similar, and only when they were measured 
at higher frequencies did the different behavior become apparent. 

lor the purpose of showing that other semitransparent mirrors dif- 
ferent from glass could be adequate to build a hybrid, a copper screen 
was used. The copper screen is an electroformed mesh 0.0005 inch thick 


I 
60.0 KMC 68.4 KMC 


{ 
65.9KMC 76.4 KMC 


PORT 4 
—_ 
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Fig. 8 — Superimposed outputs of ports 3 and 4: (a) Hybrid No. 1, 60.0 through 
68.4 kme; (b) Hybrid No. 1, 65.9 through 76.4 kme; (¢) Hybrid No. 2, 60.0 through 
68.4 kme; (d) Hybrid No. 2, 65.9 through 76.4 kme 
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with 400 square holes per square inch, with the copper separating the 
holes being 0.0175 inch wide. 

The power division of the hybrid is 1.8 to 1, and, with more screen 
reflectivity, 3 db power division should be achieved. No effort was made 
in this direction, but a semitransparent mirror of this kind should be 
weighed carefully against a glass one. 


IV. HIGH-PASS FILTER 


A TEo° high-pass filter is obtained by reducing the diameter of a 
circular waveguide. The minimum radius essentially fixes the cutoff 


frequency and the slope of the tapers determines the steepness of the 
transfer characteristics. 
The high-pass filter, Fig. 9, is made of electroformed round copper 


pipes. Two relatively smooth tapers connect the cutoff section to 7% 
inch-diameter waveguide. The cutoff section has a constant diameter of 
0.260 inch and is 1 inch long. The over-all length of the filter is 3.8 inches. 


high-pass filter; (b) exploded view. 
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The end size of the high-pass filter was selected because we already 
had on hand tapers from 7% to 2 inches in diameter.® In a final design 
of a band-splitting filter it is not necessary to pass through the inter- 
mediate size of ;’g inch, and consequently the filter can be more compact 
than the one shown in Fig. 2. 

In order to appreciate the high-pass filter behavior at frequencies 
close to cutoff, transmission and reflection were measured point by point, 
as shown in Fig. 10. In an ideal high-pass filter, power at frequencies 
above cutoff should be transmitted completely, but this implies that 
the taper should match a waveguide of a certain admittance to another 
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Fig. 10 — Transmission and reflection of high-pass filter (without tuning screws). 
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of admittance close to zero, which requires extremely long tapers. Our 
tapers are short, and Fig. 10 shows that the maximum unwanted re- 
flection above cutoff is 6.6 db. This reflection can be reduced by using 
polyethylene tuning screws. The transmission and reflection of the taper 
with tuning screws, as seen in Fig. 11, show that the maximum unwanted 
reflection has been reduced to 16.8 db without changing substantially 
the transmission characteristic. The difference between the frequencies 
at which transmission and reflection losses are below 1 db is 160 me. 
This is a very sharp cutoff. 
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Fig. 11 — Transmission and reflection of high-pass filter (with tuning screws) 
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Fig. 12 — Two-way transmission on a single waveguide using two band-split 
ting filters. 


V. ELBOW 


The elbow, as mentioned earlier, is derived from the hybrid by replac- 
ing the semitransparent mirror with a metal plate. The insertion loss is 
0.2 + 0.05 db. 


VI. ANOTHER POSSIBLE USE FOR A BAND-SPLITTING FILTER 


This filter allows using a single waveguide to transmit in both direc- 
tions. Fig. 12 shows one of the possible arrangements. Two identical 
band splitting filters are used. Calling f. the cutoff frequencies of the 
filters F, the reader can check that frequencies f < f, can travel towards 
the right and frequencies f > f. cai iravel towards the left. 


¢ 


Power leaving the repeater for f. can leak into the repeater for 
f > f. only by passing through cutoff filters, and consequently that 
leakage can be made arbitrarily small. On the other hand, power leaving 
the repeater for f > f. can leak into the repeater for f < f. because of 
unbalance in the hybrids H; and He. This leakage can be reduced if 
necessary by including between hybrid H; and the repeater a filter like 
the one shown in Fig. 13. It consists of a hybrid and two high-pass 
filters like those described previously. 


f>f; 
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Fig. 13 Filter to eliminate high frequencies. 
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VII. CONCLUSIONS 


A constant-resistance filter capable of dividing a wide band in two 
has been described. It can easily handle one octave in the millimeter 


region with only 1.5 db insertion loss for each subband. The splitting 


transition takes place in a very narrow band (160 mc). Another use of 
the filter: it allows using a single waveguide to transmit in both direc- 
tions. 

Two of the components of the filter are important devices: an elbow 
and a hybrid. Both are quasi-optical and operate easily over one octave 
in the millimeter region. The elbows allow sudden 90° turns of a 2 inch- 
diameter multimode waveguide with relatively low insertion loss 0.2 + 
0.05 db. Without them the band-splitting filter would be very bulky. 
The importance and uses of the hybrid need hardly any comment. 
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Margin Considerations for an Esaki 


Diode-Resistor OR Gate* 


By H. K. GUMMEL and F. M. SMITS 


(Manuscript received July 21, 1960) 


An Esaki diode-resistor logic, powered from a three-phase supply and 
involving oR gates, is analyzed. Practical switching times are of the order 
of 10 |\R-| C. The voltages at which the current maximum and the current 
minimum occur set an upper limit on the achievable logical gain. For a 
sum of fan-in plus fan-out of 3, the margins on key diode and circuit pa- 
rameters must be better than +2 per cent, with all margins assumed equal. 
The margins can be +3.5 per cent for a fan-in plus fan-out of 2, which, 
however, restricts the applications to shift registers, flip-flops, and the like. 


I. INTRODUCTION 


Esaki diodes are being considered for high-speed logic due to their 
potentially high switching speeds. Several papers have already ap- 
peared on the use of Esaki diodes in logic systems.':?:*4> In such sys- 
tems the bistable V-7 characteristic of the diodes is utilized to define 
two logical states (‘‘zero”’ and “one’’). The bias current of the diode, 
together with a trigger current derived from a previous stage, deter- 
mines which of the two states will be attained. If the trigger current 
can be kept small with respect to the output current, logical gain can 
be achieved. This generally requires that the characteristics of the 
diodes be well controlled, since the logical gain will primarily depend 
on the margins of the diodes and of the circuit parameters. Conse- 
quently, the considerations of margins become of prime importance in 
the design of logical systems. 

In this paper, worst-case margin considerations are given for one of 


the simplest types of Esaki diode logic, a diode-resistor logic powered 
from a three-phase supply. In particular, the discussion is restricted 
to the least complicated logical element — an or gate. 


* This work was supported in part by the U.S. Army Signal Corps under con- 
tract DA-36-039 sc-64618. 
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Fig l Esaki diode-resistor logic. 


First, a general description of the system will be given (Section I1), 
followed by a qualitative margin analysis (Section III).* In Section LV 
it is shown that the present system permits only a finite logical gain, 
even for zero margins and infinite switching time. The switching speed 
is analyzed in Section V, followed by the quantitative margin analysis 
(Section VI). The final result of the quantitative margin analysis is 
brought into a form corresponding to the qualitative analysis, which 
permits the reader to follow the discussion (Section VIL) and the con- 
clusion (Section VIII) without studying in detail the reasoning in Sec- 


tions V and VI. 


Il. ESAKI DIODE-RESISTOR LOGIC 


The basic stage in an Esaki diode-resistor logic consists of a series 
arrangement of a diode and a resistor, Ry, , with input and output cou- 
pling resistors as shown in lig. 1. The bias voltage is chosen such that, 
without any voltage at the far ends of the coupling resistors, it gives 
rise to a current through the diode which is below the peak current. 
Consequently, the diode will remain in its low-voltage state. With addi- 
tional current supplied to the center node through one or more input 
(or output) resistors, the diode can be made to switch into the high- 
voltage state. With the bias current only slightly smaller than the peak 
current, very small ‘‘trigger’’ currents are necessary. 

Once the diode is in its high-voltage state it will remain there even 
if current is now withdrawn at the node. It is only necessary that the 
current through the diode remain above the valley current /,. The 
maximum current that can be withdrawn is, therefore, the difference 
between the bias current and the valley current. The ratio of this out- 
put current to the trigger current constitutes the logical gain. 


* The authors are indebted to J. H. Vogelsong, whose unpublished margin 
studies are incorporated in this section 





MARGINS FOR ESAKI DIODE-RESISTOR OR GATE 


By proper choice of the bias current, the current of at least one input 
or the combined current of several inputs is necessary to trigger the 
diode. The diode accordingly can act as an oR gate, as an AND gate or 
as a THRESHOLD gate. The output resistors can be connected to the 
node of a subsequent stage. Similarly, the input resistors are powered 


from nodes at previous stages. The extension of this principle leads to a 


logical network. 

In such a network it is, however, necessary to determine the direc- 
tion in which information will be propagated. One elegant method 
utilizes a three-phase bias supply® as depicted in Fig. 2. Adjacent diode 
stages are powered from different phases. Thus a diode on phase a, for 
example, is triggered from a diode on phase c, and it will trigger a di- 
ode on phase B. 

Even in such a multiphase system “‘backswitching” can occur’ To 
illustrate this, consider the arrangement of Fig. 3. If the stages repre- 
sent OR gates, one stage in a high-voltage (‘‘one’’) condition will trigger 
a following stage. Stages 1 and 2 are powered from phase a, while 
stages 3 and 4 are powered from phase B. Assume that stage 1 is in the 
“one”? condition and stage 2 is in the ‘‘zero’’ condition. As soon as 
phase B is applied, stage 3 will assume the ‘“‘one”’ condition. Since stage 


, 


3 is coupled to stage 2, it will trigger stage 2 into the ‘‘one”’ condition, 
resulting in an erroneous “‘one”’ in stage 4. 

To avoid such backswitching, a system of oR gates must be arranged 
in such a way that no multiple input is fed from any stage having a 
multiple output. Logical design then forces the use of ‘‘booster’”’ stages, 
e.g., stages with one output driving stages with one input. 
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Fig. 2 Three-phase bias supply 
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PHASE C PHASE A PHASE B PHASE C 
Fig. 3 — Logie network of or gates leading to backswitching. 


Ill. QUALITATIVE MARGIN ANALYSIS 


As pointed out before, the magnitude of the trigger current depends 
on the difference between peak current and bias current. This differ- 
ence can be made small (and therewith the logical gain large) if both 
parameters are tightly controlled. For an or gate the qualitative effect 
of a spread in the parameters on the logical gain can be readily demon- 
strated. 

Assume that the peak currents J, of the devices fall in a range be- 
tween J, and J,, and that the bias currents fall in a range between 
Ts min aNd Ty max - One can then introduce such relative variations as: 

) Pe & (1) 
I, 
and 
Ly max — Lo min ‘ (2) 
I, 

With a maximum valley current J, max for all devices one can define 

a “‘valley-to-peak”’ ratio 
| - max 


max (3) 
I, 


As pointed out before, for triggering a device, the total current through 
the device must exceed the peak current. It is plausible (and will be 
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discussed in detail in the next section) that an overdrive AJ is necessary 
to ensure switching with the required speed. We thus introduce a rela- 
tive overdrive 

tam. (4) 

I, 
{In a worst-case analysis, it must be ascertained that a stage giving a 
minimum total output current is capable of delivering into each out- 
put stage a trigger current which is at least as big as the trigger current 
required in the worst-case. The logical gain, e.g., the number of stages 
(n) that can be connected to one output can be found by equating the 
minimum current that can be delivered into an output stage (out min) 
to the worst-case trigger current (/;,). Due to the bilateral nature of 
the Esaki diode (output and input are identical), this number is the 
sum of inputs plus outputs (‘‘fan-in” plus “fan-out”’). 
For these currents the following normalizations are introduced: 


Feat min > 
= —— (f 
n L, 6) 


For a qualitative analysis the magnitude of these currents is found 
readily by graphical considerations. 

Fig. 4 shows a voltage-current characteristic of an Esaki diode in- 
cluding variations. Since the maximum bias current must equal the 














VOLTAGE —> 





Fig. 4 — Qualitative margin analysis. 
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minimum peak current, the entire spread in bias currents (8) must lie 
below the spread in peak currents (7). Since for a unit with the highest 
peak current a minimum overdrive must be assured, 6 must be added to 
the maximum peak current. From Fig. 4 one obtains for the normalized 
trigger current 


r="r+Bt6, (7) 
and for the total minimum output current 

nn —py-— £B. 
Equating 7 and 7 gives the logical gain as 


| —y-—8B 
=n. (9) 


e+B+é 


An evaluation of this equation will give a first-order estimate of the 
required margins. This analysis, however, neglects the effects of the 
peak and valley voltages, which even for zero margins will limit the 
logical gain under certain conditions as will be shown in the next see- 
tion. For a full evaluation of the margin equations, the relation be- 
tween relative overdrive 6 and the switching speed must be known. This 
analysis will be given in Section V. The detailed margin analysis in 
Section VI will not only include the variables considered in (9) and the 
effect of the peak and valley voltages, but it also will include the varia- 
tions of these voltages and the variations in the coupling resistors. 


IV. LIMITATIONS DUE TO FINITE VOLTAGE LEVELS 


The diode-resistor logic discussed here has an upper limit in the logi- 
cal gain if stages that are to be driven have a fan-out larger than their 
fan-in. This limitation is determined by the magnitude of the voltages 
for the current peak and for the current minimum, and exists even if 
all error margins and the valley current are zero. 

To demonstrate this effect, consider the extreme case of a stage hav- 
ing one input and (n — 1) outputs. Assume a device characteristic as 
shown in Fig. 5. The magnitude of the bias current is determined by 
the condition that the stage under consideration is not permitted to be 
triggered into the high-voltage condition if none of the stages connected 
to either the input or the output resistor is in the high-voltage condi- 
tion. Due to the bias pulse overlap with the previous and the following 
stages respectively, the far ends of the coupling resistors can vary in 
voltage between zero and V,, the voltage at which the current maxi- 
will 


mum is reached. The coupling resistors whose far ends are at V, 
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Fig. 5 — Germanium Esaki diode characteristic with load line. 


not shunt any current, and it has to be ascertained that the bias cur- 
rent under no condition will supply a current greater than 7, to the 


diode. With only one coupling resistor acting as a shunt, the maximum 
bias current becomes 


ly I, + V;G, (10) 


where G is the conductance of each coupling resistor. 

A stage is to be triggered into the high-voltage condition at a time 
when the output resistors are at ground potential, and the trigger cur- 
rent must increase the current through the diode above the peak cur- 
rent. Since, for the case of one input resistor, (n — 1) resistors act as 


shunts, one obtains for the trigger current 
tn = ie ob (n — LGV, —_ I, =(n — 2)GV,. (11) 


It must be possible to supply this trigger current from the output of 
one previous stage which is in the high-voltage condition. The far end 
of this particular coupling resistor thus is at the “valley” voltage V, , 


with the near end at v, . One thus obtains for the output current 
| CVs * V DG. (12) 
Mquating Jou, and J,, yields 


+ 1, (13) 


This demonstrates that the sum of fan-in and fan-out for such an 
asymmetrical stage remains finite, even in the case of zero tolerances. 
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V. SWITCHING SPEED 


The speed considerations are based on a diode characteristic as shown 
in Fig. 5, which shows a good but still practical characteristic for a 
germanium diode. The combined conductances of the input and output 
resistors are shown as the load line. It is chosen in such a way that 
while touching the peak point it intercepts the valley. This choice of 
the load line will allow obtaining a maximum current output. 

Assume now that a current of magnitude J) (> J,) is applied to the 
parallel combination of diode and load resistance as shown in Fig. 6. 
The capacity represents the diode capacity (plus any shunt capacitors 
in parallel with the diode). Any series inductances have been neglected. 

During a transition from a low-voltage state into a high-voltage state, 
the capacity shunting the diode must be charged. The charging current 
at a given voltage is the difference between the supplied current Jp and 
the sum of the load current and the conductive current through the 
diode at any given voltage. In Fig. 6 this charging current 7, can be 
read off as a function of voltage, since it is just the difference between 
the load line and the static characteristic. The time required to go from 
voltage V, to voltage V, is given by 


f° ctv) av 14 
SE daa ( ) 
‘Vv. I.(V) 

A numerical integration of this equation thus can give the switching 
time between two arbitrary points. For the problem on hand, the 
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Fig. 6 Capacitance charging current 
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switching time from a low-voltage state to a high-voltage state is of 
interest. 
Prior to switching, the diode voltage corresponds to the stable point 


(Fig. 5) at low voltage. After switching, the diode voltage corresponds 


to the stable point at high voltage. Switching is accomplished by the 
application of a trigger current which lifts the load line above the di- 
ode characteristic as depicted in Fig. 6. This lifting of the load line 
moves the high-voltage intercept to the right, and the diode voltage 
will approach the voltage corresponding to this intercept. After removal 
of the overdrive, the voltage would then decrease again. Thus, switch- 
ing can be considered as completed when, with applied overdrive, the 
voltage of the stable point prior to the application of the trigger cur- 
rent is reached. 

For the analysis to be independent of the particular bias current, it 
is convenient to consider, as final voltage, the intercept of a load line 
which just touches the peak. For an analytical treatment the following 
two simplifying assumptions will be made: 

i. The voltage dependent capacity C(V) will be replaced by a con- 
stant average capacity C. 

ii. The diode characteristic will be approximated by two parabolic 
sections and a straight section. 

Let a and B (Fig. 7) be the points on the diode characteristic at which 
the slope is equal to that of the load line, and let 74 and /, be the charg- 
ing currents at these points. Let c¢ be the point at which one parabolic 
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Fig. 7 — Analysis of the switching time. 
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approximation joins the second approximation and let bp be the point 
where the straight section commences. Then the time needed to switch 
from an arbitrary point F in the low-voltage part to the final point F is 
given by 


Vo dV VD dV 


I, _ 1,(' = ) 2 aS if mat 


VP dV 
sale 


a i, —¥ 


I, - IT, —-—I1I,) = > 
* Se Vr — Vo 





C 





Here the constants describing the curvature of the parabolas are ex- 
pressed in terms of the peak current, and in terms of constants V,; and 
V2 having the dimension of voltages. Such a presentation is convenient 
since the constant V,; is equal to the peak voltage and the constant V2 
is of the order of magnitude of the difference between valley voltage and 
peak voltage. Note that V; and V. depend on the diode characteristic 
only and are independent of the load line. Performing the integration of 


(15) yields 


T vs I, (tan - + 7) 
Tee aye 


(16) 


CV. p i "D D 
+ i 4 2 (tanh a + tanh"! | +C a ne a In (72). 


The constants in this equation are defined in Fig. 7. It should be noted 
that 7, corresponds to the overdrive (J, AJ) and that /,/J, 


6 as defined in (4). For an evaluation of this equation, the value of the 
capacity C must be known. This capacity, however, can be expressed 
through the characteristic time 7» R~- | C, which time is usually con- 
sidered as the figure of merit for an Esaki diode. It is therefore possible 
to express the switching time in terms of the time 7). With such a nor- 
malization, the results of the analysis will be fairly general. 

Fig. 8 shows an evaluation of (16) giving the switching times in terms 
of the characteristic time 7» for switching from zero to the final voltage. 
The assumed value of the load line permits a maximum output current 
while conforming with the margin considerations. Using zero instead of 
a finite voltage as the starting point lengthens 7, only insignificantly. 

As can be seen in Fig. 8, the switching behavior can be fairly well 


approximated by 


t/t) = 2.25 T/T a = 2.25/56. (17) 





MARGINS FOR ESAKI DIODE-RESISTOR OR GATE 


Ts | 
APPROXIMATION To = 2.25 VAL 


CALCULATED | 
CURVE 


0.4 0.6 0.81.0 2 


Al wy per CENT 
Ap 


Fig. 8 — Switching speed. 


VI. QUANTITATIVE MARGIN ANALYSIS 


The qualitative margin analysis of Section III neglected variations of 
the coupling resistors, the effect of the peak and valley voltage, and the 
variations of these voltages. As a simplification in the complete analysis 
it will be assumed that the bias is supplied from a constant current 
source. Under this assumption the total load line of the diode is given 
by the sum of the conductances of the input and output resistors. These 
resistors will terminate at the nodes of adjacent units, and it is necessary 
to include the voltage of these nodes in the analysis. 

As in the qualitative analysis one has to find the current which in the 
worst case will trigger a stage within a desired time. This current must 
equal the minimum current that under worst conditions will be delivered 
into an output resistor. 


6.1 Trigger Current Needed 


The lower the bias current, the larger will be the necessary trigger 
current. The maximum bias current, however, must be low enough that 
the stage under consideration will not assume the high-voltage condition 
unless one driving stage is in such a high-voltage condition. The margin 
of the bias current then determines the difference between the maximum 
and the minimum bias current. 

The characteristics of all diodes in a system will show a spread, and 
will fall between two extreme characteristics, which represent the ac- 
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CURRENT => 

















VOLTAGE => 
Fig. 9 — Spread in characteristics. 


ceptance limits. Such extremes are indicated in Fig. 9. The conductance 
of the n; input resistors and the np output resistors will fall between an 
upper bound of value G, and a lower bound of value G, . 

For the following analysis one determines the point L (Fig. 9) where 
a load line of the lowest total conductance (n; + mo) G, is tangent to 
the lower characteristic, and the point u where the load line of largest 
conductance (n; + mo) Gr is tangent to the upper curve. The voltages 
and currents corresponding to these points are Vy, Vi, Jv and J, re- 
spectively. 

In a three-phase system as considered here, only two adjacent units 
will be powered at the same time. Thus one has two extreme conditions 
under which the unit under consideration should not be triggered: 

i. The unit under consideration and the previous unit are powered, 


in which case the far ends of the input resistors may be as high in voltage 


as V, while the output resistors are at ground potential and 

ii. The unit under consideration and the following unit are powered, 
in which case the far ends of the output resistors may be as high as V, 
in voltage while the input resistors are at ground. 

The maximum permissible bias current should not switch a unit if it 
has the lowest peak current and if the smallest current is shunted by the 
coupling resistors. This current is given by 


I, max I, + Nin iV 1 ’ (18) 


where Nin represents the minimum of n; or nm. The minimum bias 
current is below the maximum bias current by the spread in bias currents 
Al, 
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I, min = Ty max ~ Al, . 


In order to trigger a diode it is necessary that a current of at least 
Ivy plus necessary overdrive for speed is delivered to the diode. This 
current will be the sum of the minimum bias current plus the trigger 
current minus the current drained through the input and output re- 
sistors. However, since at least one stage has to supply the trigger cur- 
rent, only n; — 1 input stages can act as a load. Thus one obtains under 
worst-case conditions 


Ty + Al - Te + I, min ~ GuVi no (n; bi 1) Gy (Vu Vort min) L « (20) 


In this equation Vor min is the minimum off-voltage a unit can assume. 
(It should be realized that the voltage V, discussed previously can. be 
considered as Vor max -) 

To bring (20) into normalized form the following quantities are in- 
troduced 


_ Go (21) 


Vo 


Vo pit V ott min 
Vi ‘ 
_ GV 
i ne hi ° 


With these definitions and with the definitions (1), (2), (3), (4) and (5), 
equation (20) combined with (18) and (19) gives for the normalized 


(24) 


trigger current 
r= 7+8+64+ y[(1 + p)(l + o)no — Mmin + (1 + pige(ni— I). (25) 


The term multiplied by y involves the additional terms not present in 
the equivalent equation (7) of the qualitative analysis. 


6.2 Output Current Available 


In a three-phase dioc» system as discussed here, the output current 
must be available while the stage under consideration and the output 
stages are powered. The nodes of the input stages accordingly are at 
zero potential while the output stages are at the peak voltage. Certainly 
the driving stage is in the high-voltage condition and its voltage corre- 
sponds to a voltage V, in the “valley” of the characteristic. 

One thus can represent the situation by a circuit as shown in Fig. 10. 
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Fig. 10 — Derivation of output current. 


The conductances G, represent the coupling resistors and the voltages 
V. fall in the range of the peak voltages. For the analysis it is more con- 
venient to represent the voltages V, and their associated conductances 
as current sources feeding the node. This leads to the representation as 
shown in Fig. 11. Here it must be remembered that V, 0 for the input 
conductances. From this representation the valley voltage V, is readily 
obtained as 


I, + =V.iG. — I, 


D> OF 


(26) 


If G; represents one particular coupling resistor to an output stage the 
current into this stage is readily obtained: 


I (V, — V)G 
Substituting V, from (26) gives 
I, + ViG, — I, — V;2G, 


Tout ; G;. 
2G; 


This current has a minimum if 


| 1 = Ip +E GKVK 





«-—Vy 











Fig. 11 Circuit equivalent to Fig. 10. 
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It ” I, min 9 
G; = G, and all other G, = Gy, 
and if for the output stages 


V; = V. and all other V, = VL. 


Introducing these conditions into (28) and substituting, one obtains 
in normalized form: 


n{n + (n i 1 1) p] =l-—yv-Bp- yi -- p) (1 + o)n; — Nmin 
+ + pelnn — MM. 


Again, this expression has additional terms which are absent in the 


(30) 


equivalent equation (8) of the qualitative analysis. 


6.3 Logical Gain 


To determine the sum of fan-in plus fan-out, the normalized trigger 
current, (25), must be set equal to the normalized output current, (30). 
lor this it is necessary to consider the particular configuration in which 
stages are interconnected. 

For a given n , the trigger current, (25), will have a maximum if n; = 
lL and mp = n — 1. Similarly, the output current, (30), will have a mini- 
mum if n; = n — 1 and m = 1. Thus the worst combination of two 
stages is the case in which a stage with a multiple input and a single 


output drives a stage with a single input and a multiple output. Such a 


combination represents a “booster”? stage, which is an important con- 
figuration to avoid backswitching. In the following, the analysis will 
therefore be given for such a combination. At this point it is convenient 
to introduce 


n* =n + (n — l)p. (31) 


For the worst-case combination of stages, (25) and (30) take then the 
form 


r=nr+B8+6+4+ y[(n* — 1)(1 +0) — 1], (25a) 
y— BB — y[(n* — 1)(1 +04) — 1). (30a) 


Besides the variations of all parameters, these equations involve the 
value of the coupling resistors in the term y. Since the load impedance 
determines the operating point in the high-voltage condition, one can 
express y by this point, i.e., by the valley voltage and the valley current. 
Due to the variations in the load conductances and the bias current, 
this operating point must be defined for a particular combination of 
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these parameters. The combination to be chosen is the one leading to 
the minimum output current. One thus can use (26) to express y in 
terms of the valley voltage V, and the valley current /, . Introducing 
the condition (29) into (26) (and assuming the particular configuration 
under discussion) gives: 
y= fomin + Vo ~ 1, ae 
"(n= 1)G0 + Gr ee 
It is plausible (and ean be proven readily) that a device characteristic 
leading to an operating point with a higher voltage V, or a lower cur- 
rent /, results in a higher output current. Thus the operating point as 
defined by (28) is a worst-case condition; such an operating point is 
schematically indicated in Fig. 12. It specifies an area (shaded) which 
must be cleared by the high-voltage branch of all diode characteristics. 
Introducing a normalized valley voltage 


(33) 


and using all previous normalizations permits one to use (32) for the 
elimination of y: 


— l1—v-8 
pie n*y — (0 + 2)- 


Equating 7 and 7 as given in (25a) and (30a) and using the above 
expression for y leads to the final result: 
l—vy-—8 [In + (n — lL)ply — (6 + 2) (35) 
e+B+6 y-I([n+ (n — l)pl(o+1)4+1 


—_»> 


CURRENT 


























VOLTAGE — > Vv 


Significance of valley voltage and valley current. 





MARGINS FOR ESAKI DIODE-RESISTOR OR GATE 
VII. DISCUSSION 


The final expression of the quantitative margin analysis, (35), differs 
from the qualitative result obtained in Section III, (9), in the right- 
hand side only. While in the qualitative expression the right-hand side 
is the sum of fan-in plus fan-out (n); in the quantitative expression the 
right-hand side is a function of n and these additional variables: the 
relative variation in the conductances of the coupling resistors p, (21); 
the relative variation in the voltage for the current peak o, (22); and 
the minimum ratio of valley voltage to peak voltage v, (33). 

On account of the similarity in the results it is convenient to introduce 
a generalized 7% defined as 


r [In + (nm — l)ply — (o + 2) pa 
i= —_———— -——- (56) 


~~ y — [nt (n= I)pl(o +1) + 1° 


Even for o and p equal to zero the generalized *% becomes infinite for 
n=yrt1. (37) 


An infinite % imy lies zero margins for all variables and zero overdrive. 
Thus the result of Section IV is recovered. 

In considering the effects of finite margins, specific assumptions as to 
the relative magnitude of the margins on the various variables must be 
made. For the primary variables in the left-hand side of (9) or (35), the 
maximum ratio v of valley current to peak current is assumed as 0.1, 
since this corresponds to a good ratio achievable in germanium units. 
The relative overdrive 6 will be expressed in terms of the switching speed 
using the calculated relation obtained in Section V. The result is shown 
in Fig. 8, which gives the switching time r+, in terms of the characteristic 
time t> = | R~ | Cas a function of the relative overdrive 6. 

The other quantities of the left-hand side will be treated as inde- 
pendent variables. To keep the discussion fairly general, we assume that 
all significant parameters are kept within the same relative variation. 

In general, the bias current will be determined by a voltage and a 
resistor. Thus the spread in bias current 6 is the result of an uncertainty 
in a voltage and in a resistor. In worst-case analysis the two margins 
have to be added. Assuming the two margins to be equal and introduc- 
ing a relative maximum variation, x, from the center value, one can 
express 6 as: 


B = 4x. (38) 


In the analysis, no mention has been made of noise; in particular, the 
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possibility of undesirable crosscoupling between stages. Such “noise” 
most conveniently can be expressed in an equivalent relative variation 
of the peak currents. The parameter mz accordingly contains noise as a 


second variable besides the actual variation in peak currents. Equating 


these variations to the variations in the bias current results in: 


4x. (39) 


Fig. 13 shows a plot of (9) with 8 and m expressed by (38) and (39) re- 
spectively and with vy = 0.1. It can be seen from the figure that even for 
small logical gain fairly tight margins are required. It also becomes 
apparent that switching speeds below 10 79 are impractical. 

To evaluate the importance of the margins of the additional variables, 
which enter in 7, the relation (36) between actual n and generalized 7 
has to be evaluated. Fig. 14, as an example, shows this relation for an 
actual n = 3, which corresponds to the minimum sum of fan-in plus 
fan-out required in a logical network. It can be seen that with an in- 
creasing ratio of valley voltage to peak voltage (y) the limiting value of 
nr = n+ (n — I)pis rapidly approached. It also becomes apparent that, 
for sufficiently large values of the valley voltage, the spread in peak 
voltage and in the values of the coupling resistors are only of minor 
importance. 

Since for good germanium Esaki diodes y = 8 for »v < 0.1, these 
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Fig. 13 Permissible variations from nominal value of the important parame 
ter as a function of generalized 7 





MARGINS FOR ESAKI DIODE-RESISTOR OR GATE 




































































1 


7 
y 





Fig. 14 — Generalized % as a function of y = V,/V_,. 


values have been assumed in the construction of Fig. 15, which shows 
the dependence of the sum of fan-in plus fan-out (n) on the percentage 
variations x from the correct value of the bias voltage, the bias resistor 
and the peak current, and a noise equivalent current expressed as a 
percentage of peak current. Because the effects of the coupling resistors 
and the peak voltage are relatively small, a fixed relative variation of 
+2.5 per cent has been assumed for these quantities. 


VIIT. CONCLUSIONS 


An Esaki diode resistor logic with three-phase power supply shows 
several basic limitations even if it only involves or gates. 
The possibility of backswitching limits the design of logical net werks, 


requiring the incorporation of ‘booster’ stages in which a device with 


one output drives a device with one input. 

Switching times shorter than 10 | R~ | C are not practical; however, 
this is not a very severe limitation. 

The finite ratio of the voltage for the current minimum to the voltage 
for the current maximum limits the logical gain even for the case of zero 
margins. 

From Fig. 15 it is apparent that, even for a sum of fan-in plus fan-out 
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Fig. 15 — Permissible variations from nominal value of the important parame- 
ter as a function of fan-in plus fan-out for a specific example. 


(n) equal to 3, worst-case margins of less than +2.5 per cent are required. 
For an AND gate, a two-sided limit on the trigger current is required, 
making the margins even tighter. It thus appears questionable that an 
Esaki diode resistor logic with n 2 3 is practical, if operation under 
worst-case conditions is to be guaranteed. 

Only for n = 2 do the margins appear tolerable under worst-case 
conditions. However, such a value of n does not permit the construction 
of a complete logic network, and implies a restriction to applications such 
as memories, flip-flops, shift register and the like. 


REFERENCES 


1. Lewin, M. H., Samusenko, A. G. and Lo, A. W., The Tunnel Diode as a Logic 
Element, Solid State Circuits Conf., Philadelphia, 1960. 

2. Neff, G. W., Butler, S. A. and Critchlow, D. L., Esaki (Tunnel) Diode Logie 
Circuits, Solid State Circuits Conf., Philadelphia, 1960. 

3. Miller, J. C., Li, K. and Lo, A. W., The Tunnel Diode as a Storage Element, 
Solid State Circuits Conf., Philadelphia, 1960. 

4. Chow, W. F., Tunnel Diode Digital Circuitry, Solid State Circuits Conf., 
Philadelphia, 1960 

5. U.S. Navy Department, Bureau of Ships, Electronic Division, The Dener Diode 
as a Digital Computer Element — Phase 1, Project Lightning, Vol. 4, Ch. 8, 
p. 83. 

6. Goto, E.,On the Application of Parametrically Excited Nonlinear Resonators, 
J. Inst. Elect. Comm. Eng. Japan, 38, 1955, p. 770. 

7. Taylor, R. G., unpublished work 





Noncylindrical Helix Waveguide 


By H. G. UNGER 
(Manuscript received August 9, 1960) 


Small uniform deformations of the cross section of helix waveguide perturb 
the circular electric waves slightly. From these perturbations the added 
circular electric wave loss is found in a uniformly deformed helix wave- 
guide. For a nonuniformly deformed helix waveguide Maxwell’s equations 
are converted into generalized telegraphist’s equations. By an approximate 
solution for small deformations, mode conversion and circular electric wave 
loss are found, 

Random imperfections with small correlation distance cause an average 
circular electric wave loss that is nearly independent of the wall impedance 
which the helix jacket presents to the waveguide interior. It is therefore nearly 
the same as in metallic waveguide. Near 50 kme, the rms value of elliptical 
diameter differences should not be more than 0.0015 inch in order that on 
the average not more than 10 per cent of TE, loss in a perfect 2-inch inside 
diameter copper pipe is added to the TE, loss in a helix waveguide of the 
same inside diameter. 


I. INTRODUCTION 


Helix waveguide composed of closely wound insulated copper wire 
covered with a jacket of dielectric material and surrounded by a coaxial 
metallic shield is a good transmission medium for circular electric waves.’ 
In long distance communication with these waves helix waveguide is 
useful as a mode filter, for negotiating bends and particularly as a trans- 
mission line proper. The different applications of helix waveguide require 
different properties of jacket and shield. Corresponding design rules 
have been worked out.” 

The loss of circular electric waves in a metallic waveguide decreases 
steadily with increasing frequency only if the guide is perfectly round. 
The same is true for the helix waveguide. To maintain the low-loss 
properties of the circular electric wave, the helix waveguide must be 
manufactured to a high degree of roundness and uniformity. 
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As long as the guide is cylindrical, i.e., any deviation from roundness 
is independent of distance along the guide, increased circular electric 
wave loss is the only effect of such deviation from roundness. But if at 
the same time this deviation changes with length, the transmission char- 
acteristics of the guide will be further degraded by mode conversion- 
reconversion effects. At any change of cross-sectional shape of the guide, 
power of the circular electric wave will be scattered into unwanted 
modes, and vice versa. The amount of power scattered depends not only 
on the magnitude of change but also on the rate of change with length 
of these deviations from roundness. 

Two cases, that of the uniform noncircular helix waveguide and that 
of the nonuniform helix waveguide, will be analyzed separately. In the 
first case, a perturbation of the normal modes of the round waveguide 
will give a simple answer. In the second case, however, Maxwell’s equa- 
tions will be converted into generalized telegraphist’s equations,’ and 
the results appear to be much more involved. 

This paper partly represents an extension of an analysis of non- 


° ° ° . 4 ° ° 
cylindrical metallic waveguide’ to helix waveguide, and partly uses the 


results of a mode-conversion analysis which was made more recently.’” 


il. THE UNIFORM NONCIRCULAR HELIX WAVEGUIDE 


The mathematical model with which helix and surrounding jacket 
structure is represented in this analysis is an anisotropically conducting 
sheath. The sheath conducts perfectly in circumferential direction and 
has a surface impedance Z in longitudinal direction. A cylindrical co- 
ordinate system (r,g,z) will be used, in which r = 0 coincides with the 
axis of the guide. At present the inner radius of the guide is a function of 


g only: 
(1) 


The anisotropic sheath imposes the following boundary conditions a 
r = @: 
dé 


E, +E 0) 


¢ "" de 





NONCYLINDRICAL HELIX WAVEGUIDE 


The deviation from the nominal radius ap is assumed to be small and 
smooth: 


16 
5< I and «K 1. (4) 
dg 

Then the electromagnetic field can conveniently be represented as a 
perturbation of the field in the round guide of radius ay : 


E=EK,+e, 
H = Hy +h. 


(5) 


lurthermore the fields at 7 = a can be written in terms of the fields at 
r=4Q4: 
2 oy, , OE y( ao, ¢) 
Ey (ag) = E( a0, ¢) + a6(¢) - ; “ , ; 
or 


If the unperturbed field is of circular electric form with Ey. = Ep, 
Ho, = 0, then, upon substituting from (5) into the boundary condition 
(2), the Taylor series (6) can be used. The perturbation field can then 


be written in terms of the unperturbed field of the circular electric wave: 


a) ag) 


Cog = —Ad(¢) 
or 


x (7) 


The boundary condition (3) imposes an additional requirement on the 
t | 
perturbation field 
e; —ZLh,(ao). (8) 


Conditions (7) and (8) suffice to calculate the complete perturbation 
field. 

A circular electric wave that carries unit power in positive z direction 
has an electric field: 


’ 2wu Ji Xor') 
Evy — 4 ¢ 
To ad o( ko ) 


where 
ko = x00, J (Ko) 0 
and 
Bo" WME — Xo. 


Here, u and ¢ are permeability and permittivity of the waveguide interior; 
w is the angular frequency. 
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The perturbation 6 of the nominal radius is a periodic function of ¢. 
A Fourier expansion is therefore in order: 


d(g) = > 6, COS pe. (10) 
Pp 


Terms with sin pg have been omitted from (10). They would only add 
identical perturbations with different polarization. Substituting from (9) 
and (10) into (7): 


€,( Go) = maa xe 270? oe 5, COS pe. (11) 
wBo p 
The expression suggests an expansion of the perturbation fields into 
terms which individually satisfy Maxwell’s equations and have the ¢ 
and z dependence of the terms in (11). Such a field is obtained from wave 
functions 


be ApJ »(xor) Sin pee 


p 
‘ 


T tp} ee Atp\J p(xor) COs pee 


and the following formulae: 
aT) _ aT 
or rdg 


OT ») OT {p} 


Cr 


rog or ’ 


Bo AT (p) 
wu or ‘ 
Bo OT 5p) 
Wu rdg 


XooT, 

jou 
Equating e,(ao) from (13) with e,(ao) from (11) and comparing in this 
equation the coefficients of cos py, a relation between a;,) , a;,) and 6, is 
obtained: 


Bo 
awn? 


, , 
he p koa ?) + J»! ko )Atp) 
we ko 
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Another relation between a,,) and a;,) is obtained by substituting for 
e, and hy from (13) into (8): 


Xo J y( ko) arp) = 2% | xo (ha en Pe £ Jy(ls)ap |. (15) 


Jwe ; wu Ao 
Kquations (14) and (15) can be solved for a;,) and ay»; . For example: 
Ap) 
ve /2B) Zb»p p 
V von Jp(ko) ko Sp(ko) _ leo E (ke) Pp Be i 





Jwedy J ‘ ( ko ) 


Jer ko) ko? we 

With a,,) and a,,) the perturbation fields of circular electric waves are 
known as functions of the 6,’s. Thus the quasi-circular electric waves 
in any slightly deformed round waveguide can be written in terms of the 
normal wave and perturbation fields. 

The propagation constant remains unchanged and equal to 78 in this 
first-order approximation. Now it is just the effect of a deformation on 
the propagation constant and especially on its real part, the attenua- 
tion constant, which is most important. Ordinarily a higher order of 
approximation would be necessary to determine this attenuation. But 
here, as in all electromagnetic problems where the dissipated energy is 
small compared to the stored or propagated energy, the losses may be 
calculated from a lower order of approximation.’ The attenuation con- 
stant is the ratio of power P dissipated per unit length to the power 
carried by the wave: 


Pa 

2P* 
Power is dissipated by the perturbation field through the anisotropic 
shield into the wall impedance Z: 


Pa = 3 Re (7) [ eee" BO ices 


This integral along the actual inner radius of the guide is to first order 
equal to the integral along the nominal radius ay : 


. ( 7, ala 
= oF i €,€,*dy de. (17) 


Ps = 


In (9) the power flow of the circular electric wave was assumed to be 
unity. Substituting for e, from (13) into (17) and using (16), it is found 
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that each Fourier component of the mechanical deformation contributes 
a, to the total loss a: 


where 


. J (ko) F 
I a B yg ( ko ) 


p 
iZ WEMy a? p J, Ko) = J f ko ) 4 
E ko w" LE ko? J st ko ) Rf p (ko ) 


(18) 





This expression for the added circular electric wave attenuation in a 
deformed helix waveguide agrees with some obvious facts: Any deforma- 
tion of a purely reactive wall does not cause any circular electric wave 
attenuation. 69 and 6; represent changes in diameter and transverse dis- 
placement, respectively, of an otherwise round guide, The circular 
electric wave configuration is not changed by them. Consequently ao 

a, = 0. 

Equation (18) is valid for but one special case. The absolute value in 
the denominator is zero whenever the characteristic equation (61) (of 
Appendix A) for helix waveguide modes of pth azimuthal order is satis- 
fied by ky. Whenever a mode of pth azimuthal order has the same 
propagation constant as the circular electric wave, 6, , however small 
it may be, causes a substantial change of the normal circular electric 
mode that can no longer be described by the perturbation expression of 
(18). 

The propagation constant of any of the asymmetric modes, to be equal 
to JB» , requires a purely reactive wall impedance. Because of finite loss, 
practical wall impedance values will always be at least slightly resistive; 

18) will therefore be valid for all practical cases. 
lor some typical cross-sectional deviations, (18) can be simplified: 


bo represents an elliptical deformation: 


ms re 
ata + Re(Z) — - 


We 


6; represents a trifoil deformation: 





: ) 
r VW 0 
ax) = } Re (Z) | — 


Wu | ho ; bo” 6 \ 2; (20) 
12 . ‘ 7 2 / 9 
~ Ww Vv Me ay 
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6, represents a quadrufoil deformation: 


ayy 


Re (Z) 





Bo ko by" 


—_ Y P 87 —_ Be — 12 
Co" ko? $y)" ME (5 4 — k;? 
, for any multifoil deformation. 


Il. NONUNIFORM HELIX WAVEGUIDE 


Here the relative deformation 6 of the guide radius will not only be a 
function of ¢ but it will also change with z. In Appendix A Maxwell’s 
equations are converted into generalized telegraphist’s for this structure. 

The deformation 6 is first assumed to be independent of z. The fields 
in the deformed but cylindrical waveguide are represented in terms of 
normal modes of the perfectly round helix waveguide. This series 
representation for the field components is then substituted into Max- 
well’s equations. With the boundary conditions (2) and (3) and an 
orthogonality relation between normal modes of the helix waveguide, a 
set of simultaneous first-order differential equations is obtained, which 
determines the z-dependence of the coefficients of this series expansion. 
If the coefficients are chosen so that they represent amplitudes A and B 
of forward and backward traveling waves of the round guide modes, 
then the system of equations for the A’s and B’s can be written as 


lAn 
: + jhmA m = » Cans’ 
az f 


dB, : ? 
_ JhmBm +7 z Cus’ 
dz n 

If the perturbation 6 of the nominal radius is expanded into a Fourier 
series (10), then the coupling coefficients are determined by the co- 
efficients of this Fourier expansion: 


VT N. - ky mk yy » Jy | Res ) 
2 Rom kav? J (kh 


p # 0: Clom]){ pn] 


Kom kon ‘ 
Pp Q: Com] [On a Os » 


ay ‘Rhomho, 
The metallic waveguide is the limiting case of the helix waveguide with 
zero wall impedance. The normal modes of the helix waveguide de- 
generate into TE,, and TM,,. The separation constant: /, XpnMo is 
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the root of J (Kp ) = 0 for TE,, modes and the root of J,(k>.) = 0 for 
TM,,, modes. The coupling coefficients (23) reduce to ¢ = 0 for inter- 
action between TE», and TM,, modes. For interaction between TKo», 
and TE,, modes the coupling coefficients are: 

a - 
av Dhomlipn V kon? — p* 





ities A (24) 

In a nonuniform helix waveguide the coupling coefficients ¢ in (22) 
are functions of z. Then (22) is a system of first-order linear differential 
equations with varying coefficients. For small deformations and con- 
sequently small coupling coefficients, solutions of (22) can be found by 
successive approximations. To simplify the representation, the B’s of 
(22) are included in the A’s. There are then always pairs of A’s associated 
with propagation constants jh, and —jh,, and coupling coefficients je, 
and —jCnm. Thus the two equations of (22) can be replaced by the first 
alone. The transformation 


jh 


A, =e’ “E, 


eliminates a common propagation factor: 


dE,, : 
a | 


dz 
The only initial conditions of practical interest are 
E,(0) = 1, 
E,(0) = 0 for n < 1. 


A TEo; wave of unit amplitude islaunched into a nonuniformly deformed 
helix waveguide. A first-order solution of (26) under these initial condi- 
tions is: 


The first-order solution is substituted into (26) for a second-order solu- 
tion: 


n’* dt ds, (28) 


and so on. 
As a typical example, a TE»; wave will be launched into a waveguide 
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that has a constant deformation 6 between z = 0---/ and is round 
everywhere else. The waveguide is thus uniform except for two dis- 
continuities at z = 0 and z = |. The wave amplitudes at any point z > l 
are, from (27) and (28), 


9 


E\(z) =1-j 0 7 5 (Car — Ian) + fle" — 1)], (29) 
n (Nae n)~ 


. Cy i(hy—hy)l , 
E,(z) = ——— («™ =» 3). (30) 
hy — hy, 
The converted wave amplitudes /,, may be regarded as being generated 
from the TE»; wave at the two discontinuities z = 0 and z = /. Then the 
conversion at one such discontinuity is: 


» 
Le. 
4n 


~ (31) 
E 


From (23) and (31), with 6 = 69, a formula for mode conversion be- 
tween circular electric waves at diameter changes is obtained: 
Eon Komkon 


— = 3 bo . 
Kom ay V/ homhon (hom ie hon) 





Likewise, a formula for mode conversion in offsets of helix waveguide 


with 6 = 6, cos ¢ can be written down. In the case of Z = 0, the formula 
describes mode conversion at offsets of a metallic guide: 
En Komkin kin 


== = a 8, 
Eom ay V 2WRomhin (Rom as hin) V kin? =_— ] 


Thus, from (31), mode conversion at an arbitrary discontinuity in helix 





waveguide can be calculated. 
Mode conversion at an arbitrary nonuniform deformation of the helix 
waveguide, however, is found from (27). 


IV. TOLERANCES OF HELIX WAVEGUIDE FOR CIRCULAR ELECTRIC WAVE 
TRANSMISSION 


The all-important question may be asked now: What deformations 
can be tolerated in a helix waveguide without any excessive degradation 
of the TE 5; transmission characteristics? There are two factors which 
degrade the TE»; transmission: (a) Additional normal mode loss in a 
deformed helix waveguide, as calculated in Section IT and described by 
(18), increases the overall TE ; transmission loss. (b) Mode conversion 
and reconversion in nonuniform sections of helix waveguide, as cal- 
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culated in Section III and described by (27) and (28), cause mode con- 
version loss and reconversion distortion of the TE», characteristic. 


4.1 Normal Mode Loss 


The normal mode loss of a uniformly deformed waveguide will be 
considered first. Helix waveguide in current experimental use at the 
Bell Telephone Laboratories has a nominal inner radius of a) = 1 inch. 
A median frequency of the planned operating range is 55.5 kme. To 
optimize various transmission characteristics, the surrounding jacket 
has been made to present a real wall impedance to the interior that is 
half of free space impedance Z = 3+/y/e. For these values, expres- 
sions (19), (20), (21) for the added circular electric wave loss have been 
evaluated: 


adh = 3.64 6”, 
a3) = 0.458 4,7, 
ayay = 0.516 64. 


By far the largest losses are caused by an elliptical deformation. The 
theoretical loss of TE; in a perfect copper waveguide of 2-inch inside 
diameter at 55.5 kme is 

ago = 2.77 X 10-*. 
In order that the increase of attenuation be not more than 10 per cent 
of this theoretical loss, the elliptical deformation should be 

6. < 0:216 X 10. 
The elliptical diameter differences in a 2-inch helix waveguide should 
not exceed 1 mil. This is quite a strict requirement. 


It is interesting to compare these figures with losses in a deformed 
metallic waveguide: 


Bo Ao 
— 2 


be", 
108, ao°53;, 


] 5By ao 


ay 


In a metallic waveguide it is the trifoil deformation which causes most 
loss. In order that such a trifoil deformation not cause more than 10 
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per cent of the theoretical TE; loss in a 2-inch metallic waveguide at 
55.5 kme, this deformation should be: 


63 < 3.42 X 10°. 


1.2 Mode Conversion Loss 


Equations (34) and (35) describe the added TE; loss correctly only 
in a waveguide with uniform, z-independent deformation 6. When the 
deformation is a function of z, as is the case in an imperfect waveguide, 


the general expression (28) describes the transmission. Changing the 
order of integration in (28), a more suitable form is obtained: 


u 


. - sfthebe de * ii 
E,\2) =1-— 5 / ot ae | ee ae ee 
n 0 


¥0 


The loss can be expressed in terms of the geometrical imperfections 6 
with ¢;, = C,6. For sufficiently small 6, 


A, 


with the loss 


= dant P,, cos AB,u — Q, sin AB,u) du 


n “U0 
6(s)6(s + u) ds, 


where 
P, + JQn 
and 


J(hy — h,) Aa, + jA8B, . 


In general, the geometric imperfections will not be known, only their 
statistical properties. Rowe and Warters® have determined with a rela- 
tion like (36) the statistics of the loss in terms of the statistics of the 
guide imperfections. Use of their analysis is made here. 

The deformation is assumed to be a stationary random process with 
covariance R(u) and spectral distribution S(¢) 


Riu) = <6(z)6(z + w)>, 


S( ¢) Reawye ease dy. 


a 


In (38), <x> is the expected value of x. 
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Taking the expected value on both sides of (37), the average added 
loss is obtained in terms of the covariance R(u) is 


et 


tA> = 2 e**"R(u)(z — u)(P, cos AB,u — Q, sin AB,u) du. (40) 
= 40 
For the following analysis, a special form for the covariance must be 
assumed. Since existing experimental information is rather vague, 
Rowe’ assumes R(u) to be exponential as reasonable physically and to 
simplify the calculation 


nay = 


Then the spectral distribution of 6 becomes 
So 


ate (42) 
1+ (Loft)? 


S( ¢) 
where S(¢) is nearly flat with spectral density So for mechanical fre- 
quencies in distance smaller than 


l 


ae (43) 
Le 


At f» the spectral distribution is down 3 db and falls very rapidly above 
fo ; Lo may be regarded as the cutoff mechanical wavelength. 

Substituting (41) for the covariance in (40) and performing the in- 
tegration over a length z > Lo, the average added loss is: 


: a P,(2r - Aa, bo) es Q,ABnLo 


<A> = SoZ ae - 
AB,,? Le? + (27 —_ Aa, Lo)? 


For Aa, Ly > 2a, (44) reduces to 


rSo2 > —P, Aan — QrABn 


<A> ~ r 
Lo n Aa,” + AB,” 


> 


or with 
g 


; 


the added average loss is for this special case: 


<A> 2 —C, " 
Os 2 OR tent (45) 
Z q(hy — hy) 


K n 


As seen from (29), a long waveguide with a uniform deformation 6 = 
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+/ <62> would have the same added loss. Equation (45) then is the added 
normal mode loss. It is also much simpler than that described by (18). 
But only when the differential loss Aa of every single coupled mode is 
very large in the cutoff mechanical wavelength Lo will the added loss be 
described by (18) with 6 = +/ <6». 

The Lo for waveguide deformation is probably small, certainly not 
much larger than | foot. Certain coupled modes might have a very high 
differential loss per foot, but then there would always be coupled modes 
with low differential loss. 

Consequently, the condition leading to (45) is not satisfied for cross- 
sectional deformation in helix waveguide. Expressions (34) cannot be 
used to determine cross-sectional tolerances. As shown by Rowe,® this 
conclusion is true for a wide class of covariance functions. 

The correct expression for mode conversion in helix waveguide is (44). 
Written as added loss per wavelength, it reads: 


</ 2 Pot aa mn A n 4) Lng ( nV nls0 
‘> Le TI ~ a, 1 ),A8,1 


Zz n AB,,7 Lo" +- (2r = Aa,Lo)? 
lor real coupling coefficients in a lossless structure, (46) reduces 
<A> 2 a... 
i — 
z n tor” + AB, 7 Le? 
and for very small Ly from (46) 
<A> 2 Lo , 
—* o> 5 LU Pa (48) 


lor a very short correlation distance, however, a more general ex- 
pression than (48) for the average added loss can be found. In this case 
R(u), whatever function it may be, has substantial values only in the 
immediate vicinity of u = 0. Then, instead of (40), 


<A> = 2 [ Ru) du . Pa 


- 0 


and, with (39), 


<A> ’ 
1S8(0) DU Pa (49) 
for any spectral distribution S(¢) of geometric imperfections with small 
correlation distance. 

Equation (47) has been evaluated in Appendix B for cross-sectional 
deformations in a helix waveguide with an infinitely high wall impedance. 
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Fig. 1 — TE»: loss in round waveguide with random ellipticity; 2 inch inside 


diameter, at 55.5 kme 


This particular helix waveguide design minimizes circular electric wave 
loss and mode conversion in bends.? In Fig. 1 is plotted the average 


ellipticity +/ <6.2> as a function of the correlation distance Lo for an 


additional average loss equal to 10 per cent of the TE; loss in a perfect 
copper pipe. For comparison, the same curve is plotted for Z = 0 repre- 
senting metallic waveguide. 

Both curves coincide for small values of the correlation distance and 
differ only slighly over the practical range of Lo. Though Fig. 1 is only 
drawn for a particular helix waveguide and a particular set of covariance 
functions, it is fairly safe to generalize: Random ellipticity of the cross 
section causes nearly as much average circular electric wave loss in helix 
waveguide as it does in metallic waveguide. 

A more exact statement has been made for the case of vanishing cor- 
relation.* When Ly is small enough for (48) to be valid, the average 
added TE, loss is independent of the wall impedance and the same as in 
metallic waveguide. 

Manufacturing imperfections usually have a small correlation distance. 
Therefore helix waveguide has to be manufactured to as close cross- 
sectional tolerances as metallic waveguide for circular electric wave 


transmission. 


V. CONCLUSIONS 


Cross-sectional deformations of the helix waveguide perturb circular 
electric wave propagation. In a slightly but uniformly deformed helix 
waveguide circular electric waves propagate with slightly changed field 
pattern. Power is dissipated into the helix jacket. Consequently, the 
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added circular electric wave loss in a uniformly deformed helix wave- 
guide is considerably larger than it is in a copper waveguide of the same 
uniform deformation. 

Nonuniform deformations cause mode conversion and added TE», 
loss. Manufacturing imperfections are expected to be random deforma- 
tions with small correlation distance. Such imperfections increase the 
average circular electric wave loss nearly independently of the wall 
impedance which the helix jacket presents to the waveguide interior. 
The average added loss is therefore nearly the same as it is in metallic 
waveguide with the same imperfections. For example, ellipticity was 
assumed to be a stationary random process along the guide with ex- 
ponential covariance. Then, even at a correlation distance of 1 foot, the 
added average TE ; loss at 55.5 kme in a 2-inch inside diameter helix 
waveguide of infinite wall impedance is only 16 per cent smaller than it 
is in metallic waveguide. 
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APPENDIX A 


Generalized Telegraphist’s Equations for Deformed Helix-Wavequide 
Maxwell’s equations in cylindrical coordinates (r,¢,z) are: 


1 dk, dk, ; 
P a ae ? —jupH, 5) 

i dy Oz 
OE, OE, 
— — —— = —jwpH, , 
02 or 
1 d(rE,) 1 OF, 
, oF r 0g 
laH. aH, 
r OZ Oz 
OH, odH 


Oz or 


—jwpnH ’ 


qwek, , 


Jwe EK, 9 


jek, . 


Ld(rH,) — 10H, 


T or 7 0g 
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The electromagnetic field in the helix waveguide can be derived from 
7 . . ry’ 4 y/ bd 
two sets of wave functions 7’, and 7’, given by 
Tr = Nod p(xnr) sin pe, 


Tn = Nyd (xn) Cos pe. 


, . . . 
The 7, and T,, satisfy the wave equation 


lj oa ue i 4/1 0T hi, id 
rior or 0g \r de 


where x is a separation constant which takes on discrete values for the 
various normal modes. The transverse field components are written in 


terms of these functions: 


. OT, oT : 
ie ie ) 


rog 


> V oT n ] aT, 
n Be ’ 
rog rog 


aT’, h,? dT. 
a a, «2 ae = Wy eee ), 
n rdg Or 


oT, h,? aT, 
rs i i I 
=> (5 "2 ri ) 


rog 


Substituting from (58) into (55) and taking advantage of (57), an ex- 
pression for the longitudinal electric field is obtained: 


E. = jou >, I, T.., (59) 


where k is the intrinsic propagation constant of the waveguide interior; 
d, and the propagation constant h, are chosen so that the boundary 
conditions of the round helix waveguide 


E,(ao) (), 
EB (do) —ZH (ao) 
are satisfied by the individual terms of (58). Only then do the individual 
terms of (58) represent normal modes of the helix waveguide. 
From E,(a9) = 0: 
OT, 
rag _ pd (kn) 
oT, Kind (Kn) 


0 r a 


(60) 
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where k, = xn@o. The prime at the Bessel function denotes differentia- 
tion with respect to the argument. The remaining boundary condition 
between FE, and H, leads to the following (characteristic) equation: 
/ 9 9 ‘ ‘ 
1 J,(k.) pha J p(Kn) | ae: 


kin Jp(Kn) end? J, (Ten) 


The characteristic equation, together with 


(61) 


wedoZ ; 


k,? -_ (k? es h,?)ac?, 
determines the separation constant k, . The transverse field components 
of any two different modes are orthogonal to each other in that: 


l 


Valm [ (Eu x H tm) dS = 


/ / 9 ; 
€ OT, OT OT i Or. 
< “a + dy, 7 — + din ct Slee 

Js €0 or rog or k? rdg 
, 9 , 
oT oT OT. i Gr 

+ — aa d, P = - a diy ey 4 = ds = on ; 

rdg or rog kK? or 


where 6, is the Kronecker symbol. The integration is to be extended 
over the cross section of the waveguide. For n = m equation (62) de- 
termines the normalization factor: 


9 : n p 7 : 
ee Ek D'(kn? — p)¥,2 + 


Vr J »( ee ) h Ve 


+ ka (1 = P 9 
kay 


y, = 2 ha) 
wae © 2) a 


with 


All quantities in (56) and (58) have now been determined except the 
current and voltage coefficients. To find relations for them the field 
components from (58) are substituted into Maxwell’s equations and 
these then are converted to generalized telegraphist’s equations. 


Add 
0 ie ] h %y OT » 
rdg " k? Or 


9 ; 

0 7 m { ] h ~~ 0 a 
T Om 9 4 

or kK? rdg 


times (50) and 
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times (51) and integrate over the cross section. The result is: 


LV wa me Ag 
Tj - 


dz WE 


(grad E.)( grad T,,,) dS + d, 73 (grad F,) (flux T,,) dS 
. ' (64) 
ci | (grad T,)(grad T’,,) dS 


8 


(grad 7’,,)(flux T,, ) as ; 


where the gradient and flux of a scalar are defined by: 
a 4 a. a 
grad, 7 —. grad, 7 —, 
or r 0g 
1 aT a oT 
. flux, 7 —~ 


flux, 7’ : 
r 0g or 


After partial integration on the right-hand side of (64), 


9 


dV 


; : (66) 
OT = Lhe Ole 
- + — ay dg 


or a k? do 


In special cases when the helix waveguide degenerates into a perfectly 
conducting metallic waveguide, the individual terms for /, in (59) are 
zero for r ay, While FE, itself, because of the boundary condition (3), 
is different from zero. Then (59) is a nonuniformly convergent series, 
which describes /, only in the open interval 0 < r < ao. Term-by-term 
differentiation will make the series diverge. Therefore the series had 
not been substituted for /#. in (64). In (66), (59) may now be substituted 
in the integral over the cross section. In the line integral, /, from the 
boundary condition (3) may be substituted. The fields at r = a can by 


a Taylor series be written in terms of fields at r = ao. Neglecting higher- 
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order terms: 
7 ‘ J OH (ao) . é‘ dé 
Ea) = —Z| H,(ao) + —2—- ad + H,(ap) 
or de 
OEAa) . 
———— a. 


or 


Thus, instead of (66): 


Vn, ie 
+ J “ Py: Fos 
dz we 
o2r sy - aT j 2 am’ 
aig | | as (E, + ZH.) + zn, | | 11 my i him 1 "| ii 
0 or dg kh? de 


lor the other of the two sets of generalized telegraphist’s equations, add 


or ao 


or 


/ 

Or a. 07 - 
— + - 

r og 


times (53) and 


r og 


L aT’, av, 
_ : — . — 
or 


times (54) and integrate over the cross section. The result is: 


ll, —" 
l + Jwet m 
dz 


— (grad H,)(flux T,,) dS + dy | (grad /7.,)( grad T,,) dS 
(69) 


“8S 
) 


' : Xa 
+ Jwe z. \ n d, ke 
[(grad 7, ) (flux T,,) — d,(grad T,,) (grad T,,)| dS. 


After partial integration on the right-hand side of (69), 


dT», ° , 2 t may! ’ 
+ Jwe J m ~ Gain H,1 mas 
dz /8 7 
oe (70) 
Y “sv a 4 ; ; 
at Jwe = Vp dy, dm =— x - | 7 n , oe dS 
n k? “8s 

To replace H, , substitute , from (58), in (52), multiply (52) by T,, 
and integrate over the cross section. The series (58) for EF, is nonuni- 
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formly convergent and cannot be used in (52). After partial integration, 


windioa [ H.T,, dS = | 


. sm’. « , +. ’ - 
E,1 mAyde + :o J n AnXn 1 n 1 m dS. (4 1) 
n “8S 


With the boundary condition (2) as Taylor series at r = ao : 
di dE ,(ao) 


E,(a.) = —E,(ao) 
: dy or 


od. 
Equation (70) can be written as: 


dl» . , . Am my * , IN, alee m 
+ Juels — 9 —_ Qo (z,: -+ : Ms a) , i de. (72) 
J0 


dz Ay dy or 


Partial integration on the right-hand side, 


’ ’ . : e te] E, ‘yy , ’ 0 ... 
E r ” = = | 0 (2 7 m + E. 7 ) dy, 
Oo 


“0 “0 0g ¢ 


and substitution of the series expressions (58), 


0k ; OR, " 2. pny! 
—-— + a — = 7 V Gar, Gt ws 
(a) or n 

reduces (72) to 


9 


= + jweVn = J Viidudmxnxm | TrT nd de 
az ) J0 
” (73) 
? - Ao 2 - - 
+ J — dnxm | E, 


“ry” , 
ot me 
Wy /0 C6 Ma 


ie. 

The interest is limited here to the propagation characteristics of 
circular electric waves. Therefore, only terms that describe direct inter- 
action between circular electric and other waves need to be retained in 
(68) and (73). When V,, and /,, are voltage and current amplitudes of 
circular electric waves, then 7, and 0T7,,/dg, and consequently the 
right-hand side of (68) and the last term on the right-hand side of (73), 
are zero. When V,, and /,, are amplitudes of other modes, then the same 
terms in (68) and (73) are zero, since FE, , H, , E, and H,(ao) are zero for 
circular electric waves. Thus (68) and (73) reduce to: 


d\ my j Rm ae 
dz We 


a sees keke? (2% 9 > 
bel, = =F Vas | v.75 de. 


Wud” “0 


(74) 


» 
& 
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The generalized telegraphist’s equations represent an infinite set of 
coupled transmission lines. It is convenient to write transmission line 
equations not in terms of currents and voltages but in terms of the 
amplitudes of forward and backward traveling waves. Thus, let A and 
B be the amplitudes of the forward and backward waves of a typical 
mode at a certain cross section. The mode current and voltage are 
related to the mode amplitudes by 


V = V/K(A + B), 
l 
-(A — B), 
VK 
where K is the wave impedance 


him 


WE 


| (76) 
If the currents and voltages in the generalized telegraphist’s equations 
(74) are represented in terms of the traveling-wave amplitudes, after 
some obvious additions and subtractions the following equations for 
coupled traveling waves are obtained: 
dA» 
dz 


dBy —_ imBm = +] 2 Cnm\ Ay + B,, ). 


dz 


+ jhmAm _ —j ps Cont Me + Be); 


The c’s are coupling coefficients defined by: 


2 2 ple 
dun $7 Bla dade Ee | TAT 8 de. (78) 
ka? “0 
To replace the d’s and T’s in (78), the customary double-subscript 
notation for the various modes in round helix waveguide is used. Then 
from (66), (70) and (73) the interaction between circular electric waves 
and other waves in deformed helix waveguide is described by the cou- 
pling coefficients: 


p x 0: ClOm] [pn] 


4 = 2 2x 
/ h pn Komk pn Pp J, ( - 


-— 6 cos pe dg, 
Rom ka¢? 2 4 J > (Kpn) /0 ellie 


N, A/ 
4 — \ 
(79) 

Komkon 


p= QO: Clom](0n) = 3-7 5 
ao V hombhon 
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APPENDIX B 


Nonuniform Helix Waveguide with Infinite Wall Impedance 
For Z — the characteristic equation (61) reduces to 
k 
ok (SO) 
ph, 


or the two equations: 


7 V } OR a 


2 


a y i: 2a0" 


For /, < ka, an approximation for the roots of (81) is furnished by 
J oat(kn) = 0, 
(82 ) 
ee: (kn) 0. 


} 


Equation (81) can be expanded about the roots of (82) to improve the 


approximations for k 
Substituting (80) for Y,, in (63) reduces the normalization factor to: 


chante — (1 _ p - = ) (83) 
V kad p( kn) h? ae kh,,ac? : 


Hence the coupling coefficient is, from (23) 


KOml p 4 p ’ 6, 
£ — — a I , Pp (84) 
2 Vv homh» ) v ag kh,,ac? Ao 
In (84) all the subscripts have been included to identify the coupling 
coefficient properly. 
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Normal Modes and Mode Conversion 


in Helix Waveguide 


By H. G. UNGER 


(Manuscript received August 9, 1960) 


Helix waveguide, composed of closely wound insulated copper wire cov- 
ered with an absorptive or reactive jacket, transmits circular electric waves 
with low loss. Mechanical imperfections, such as curvature and deforma- 
tion, cause coupling between the circular electric waves and unwanted modes 
and degrade the transmission. In designing a helix waveguide for a partic- 
ular application, a jacket must be found that minimizes the transmission 
degradation. Unwanted mode characteristics and their coupling coefficients 
must be known; these quantities are given by the roots of a transcendental 
equation involving complexe Bessel functions. 

A program has been set up for automatically finding the complex roots 
by iterative approximation. Starting from the known roots at infinite jacket 
conductivity, the characteristic equation is solved for all practical values of 
wall impedance of the jacket and all modes of interest. The representation 
of the mode characteristics as a function of wall impedance leads to a definite 
designation of modes in heterogeneous waveguide. The TE, modes of helix 
waveguide with n # 1 can have only a limited attenuation. These limits 
determine the design of mode filters. Manufacturing imperfections in- 
crease the average TE loss independently of the wall impedance. Random 
curvature with large correlation distance is produced by laying tolerances, 
but its contribution to the average loss is minimized in a helix waveguide 


with very large wall impedance. 


I. INTRODUCTION 


Helix waveguide, closely wound from insulated copper wire and cov- 
ered with an absorptive or reactive jacket, is a good transmission me- 
dium for circular electric waves.' In long distance communication, wave- 
guide can be designed to act as a mode filter, to negotiate bends or, 
particularly, to serve as the transmission line proper.? 

As in metallie waveguide, the loss of circular electric waves decreases 
steadily with frequency only in a perfect helix waveguide. Any curva- 


Onn 


avo 
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ture of the guide axis, deformation of the cross section or deviation of 
the winding from a low and uniform pitch adds to the loss and degrades 
the transmission characteristics.’ 

In a perfect helix waveguide circular electric waves propagate un- 
disturbed. Imperfections cause coupling between circular electric waves 
and other modes. Power is lost by conversion to unwanted modes and 
reconversion distorts any smooth transmission characteristics. 

In order to control mode conversion and reconversion in helix wave- 
guide with practical imperfections, and also to design helix waveguides 
for mode filters and intentional bends, the unwanted mode character- 
istics and unwanted mode coupling must be investigated. Earlier calcu- 
lations have resulted in a characteristic equation which implicitly de- 
termines the properties of helix waveguide-modes,! and also in explicit 
expressions for various coupling coefficients.2** Numerical evaluations 
of these equations have been very informative. They were, however, 
not complete enough to reveal all the unwanted mode properties and 
could not serve as a basis for helix waveguide design in every applica- 
tion 

The results of a more exhaustive numerical evaluation of helix wave- 
guide equations will be presented here. In a few typical examples these 
results will be applied to helix waveguide design problems. First the 
equations which describe wave propagation in perfect and imperfect 
helix waveguide will be listed. 


Il. PERFECT HELIX WAVEGUIDE 


A helix waveguide (Fig. 1) will be called perfect when the helix forms 
a straight circular cylinder and is wound with a low and uniform pitch. 
The mathematical model which then replaces it is an anisotropic im- 
pedance sheet at radius a conducting perfectly in circumferential direc- 
tion but with a wall impedance Z in axial direction. The Z replaces the 
jacket surrounding the helix and takes into account the finite size of 


helix wires. The electromagnetic field components in a cylindrical co- 


ordinate system (r,g,z) are then subject at r = a to the boundary con- 
ditions 


Solutions of Maxwell’s equations in cylindrical coordinates are Bessel 
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Fig. 1 — Helix waveguide and boundary conditions. 


functions J, of the radius, trigonometric functions of the azimuth and 
exponential functions of the axial distance: 
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where p is the azimuthal order of the wave. The axial propagation con- 
stant y and radial propagation constant k/a are related with the in- 
trinsic propagation constant wV ve of the material filling the waveguide: 


(‘) , wpe +. 7’. (4) 
a 


When the boundary conditions (1) and (2) are imposed on the solutions 
(3) of Maxwell’s equations the following characteristic equation results: 


kJ p(k) J p(k) 
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Values of y that satisfy the characteristic equation (5) are the propaga- 
tion constants of normal modes of the perfect helix waveguide. They 
describe wave propagation in a perfect helix waveguide completely. 
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Ill. IMPERFECT HELIX WAVEGUIDE 


Wave propagation in imperfect helix waveguide has been described 
by generalized telegraphist’s equations.2** In a perfect helix waveguide 
a normal mode n of amplitude | Z,, | propagates independently from all 
other modes m: 


dE, 


-~= —»y Ff. , 6) 
dz ¥ ” 


Imperfections cause interaction between modes so that the wave ampli- 
tudes are mutually coupled: 


dE, , ; , 
= = —Va E,, _ J se Case h in . 
az m 


The coupling coefficients are determined by the kind and size of the im- 
perfection, but they are also strongly dependent on the wall impedance. 

For circular electric wave applications, only coupling between these 
and other waves is of interest. Coupling coefficients of typical imper- 
fections in helix waveguide will now be listed. The subscript m will refer 
to the TE», wave; n will refer to any of the coupled modes. A normali- 
zation factor 

Vr J p( Kn) 
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is used to render the coupling coefficients symmetric, i.e. 


= Cmn - 


3.1 Curvature* 


There is only coupling between circular electric modes and modes of 
first azimuthal order in a curved helix waveguide: 
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where FR is the radius of curvature. 
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3.2 Deformation of the Cross Section® 


The radius a, of a deformed guide of nominal radius a can be written 


=a(l + bo 5, COS pg). (12) 
p 


Each component 6, will cause coupling between circular electric modes 
and modes of azimuthal order p: 
Kmben™ 
Nef, 
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3.3 Irregular Helix Winding‘ 


In a perfect helix waveguide the angle between a helix wire and the 
cross section is small enough to be regarded as zero. In an irregular 
winding this angle can be written 


- 6, sin pg. (14) 
Pp 


Kach component @, causes coupling to modes of azimuthal order p: 
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IV. NUMERICAL EVALUATION 


The propagation constant of normal modes in helix waveguide is, by 
(4) and (5), only implicitly given as a function of frequency and wave- 
guide parameters. The problem is to find the complex roots of a tran- 
scendental and complex equation. 

With (4), y can be eliminated from (5). Then, for a given frequency 
and guide radius, the characteristic equation determines / as a function 
of Z: 

F(k,Z) = 0. (16) 
For Z = 0 the characteristic equation degenerates into 


Jp(k) = 0, J, tk ; (17) 


the roots of which correspond to TM and TE waves respectively of 
metallic waveguide. 

Starting from the known roots of (17) for Z = 0, the solutions of (16) 
for helix waveguide can be traced by gradually increasing the wall im- 
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pedance. If ko and Zp are a known solution of (5), then an approximate 
value for the solution at Z; = Zo + AZ is given by: 


0 
7 [F( ko, Zo)|\ 
ky = ko = seein AZ. 


ak { "(ko . Zo)| 


A better approximation is found by Newton’s formula: 
F(ki, Z1) 

0 ; 

— [F(ki, 21) 

ak | 9 l | 


k — 
' (19) 


For further improvement, the process (19) can be repeated to any de- 
sired accuracy. 

The final result is the starting point for the next root at the neighbor- 
ing value of wall impedance. For the numerical evaluation, the wall 
impedance was related to the impedance of free space Zo) = Vu/e: 


(20) 


The solutions were traced along lines of constant phase ® of Z. The 
increment Ap was varied and kept sufficiently small to insure continuity 
of the process. 

The evaluation was programmed by Mrs. C. L. Beattie for automatic 
execution on an IBM 704 Data Processing System. 

The characteristic equation was evaluated for all wall impedances 
with passive phases and amplitudes up to 5000 ohms. All those solu- 
tions were traced which for zero wall impedance start as the following 
metallic waveguide modes: 


TEn, TMn , TEx», TMi, TEs , TMs. 
TEa , TMa , TE22 , TM , TH; . 
TEs , TMs, TEs: , TMs: . 


For some special wall impedance phases the evaluations were extended 
over many more modes. A value of a/A = 4.7 was assumed correspond- 
ing to a center frequency of the proposed 35 to 75 kme frequency band 
for the 2-inch inside diameter waveguide system. 

The numerical results were also used to calculate from the separation 
constant k, the propagation constant y in its real and imaginary parts. 
Figs. 2 through 6 are plotted from these results. These diagrams show 
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Fig. 2 Propagation constant y = aa + j(Bo. + AB)a in helix waveguide of 
wall impedance Z; contours in y-plane of constant magnitude p and phase angle 
® of Z/Zo ; a/d = 4.7, p = 1. 


contour lines of constant phase ® and constant amplitude p of the wall 
impedance drawn in the complex plane of propagation constant y. The 
scale on the Ba-axis has been shifted by the TE»; phase Bya = 29.305 
and represents the difference in phase constant between TE» and the 
plotted mode. 


Kach diagram is for a particular value of p, specifying the respective 
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enlarged portion of Fig 2 


azimuthal order of the modes. Figs. 2, 5 and 6 show the full range that 
is of practical interest in the present helix waveguide application. Figs. 


3 and 4 are enlarged portions of the y-plane showing in the respective 


area more details of Fig. 2. 

Fig. 3 is a good example to discuss the general characteristics of the 
modes. It gives the propagation constant of the lower-order modes with 
p 1. The curves of constant phase have their origin for zero wall im- 
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pedance on the imaginary axis of the y-plane corresponding to the phase 
constant of the particular mode of metallic waveguide. For any reactive 


wall impedance the propagation constant remains purely imaginary. It 
moves upwards on the 8-axis for inductive and downward for capacitive 
wall impedances. For infinite wall impedance the curves of constant 
phase of the wall impedance meet with those which originated at neigh- 
boring points of zero wall impedance. For example, the inductive branch 
of TM, meets with the capacitive branch of TE, . 
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Fig. 4 Knlarged portion of Fig. 3. 
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Curves for wall impedances with complex phase fan out into the 


y-plane. Some return to the imaginary axis; others continue more and 


more out to ever increasing values of the attenuation constant. 

The propagation constant is a multivalued function of the wall im- 
pedance. For any one wall impedance value there are as many different 
values of the propagation constant as there are points of zero wall im- 
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Fig. 5 Propagation constant y = aa + j(Bo. + AB)a in helix waveguide of 
wall impedance Z; contours in y-plane of constant magnitude p and phase angle 
¢ of Z/Zo;a/rX = 4.7, p = 2. 
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Fig. 6 — Propagation constant y = aa + j(8o + As)a in helix waveguide of 
wall impedance Z; contours in y-plane of constant magnitude p and phase angle 
¢ of Z/Z,;a/X = 4.7, p = 3. 


pedance on the 8-axis. Each value of propagation constant corresponds 
to a normal mode. The designation of these modes is not as simple as in 


metallic waveguide. The modes of helix waveguide are, in general, neither 
transverse with respect to any field component nor is their radial order 
well defined. Therefore, the simple designation of metallic waveguide 
TE,, or TM,, loses its significance. Nevertheless, the mode designation 
of metallic waveguide can be extended to helix waveguide or, for that 
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matter, to any heterogeneous waveguide, when the y-plane is divided 


up so that in each region the propagation constant is a single-valued 
function of the critical guide parameter. In the present case the wall 
impedance is the critical parameter. 

The dividing lines in the y-plane will be branch cuts of y in the Z-plane. 
They separate the infinite set of branches of y from each other, each 


branch corresponding to a helix waveguide mode. The branch cuts of 
y should connect the branch points in the Z-plane. The branch points 
of y in the Z-plane are saddle points of Z in the y-plane. Branch cuts 
of y should therefore go through the saddle points of Z in the y-plane. 
As many branches will be in contact at the saddle point as is the order 
of the saddle point. From inspection of the diagrams all saddle points 
are found to be of second order; therefore, only two branches of y are 
in contact at these saddle points and only one dividing line or branch 
cut must be made through each. 

The remaining path of the branch cuts is arbitrary. They should 
conveniently follow a course that never cuts contour lines of constant 
phase of the wall impedance and ends either in infinity or on the 6-axis 
at the points of infinite wall impedance. 

For example, the branch cut between TE, and TM, starts in Fig. 2 
at the corresponding point of infinite wall impedance and separates the 
contour line (® = 40°) coming from TE, from the contour line (® = 
15°) coming from TM,,. Somewhere in the y-plane the dividing line 
hits a saddle point of Z = f(y). Beyond this saddle point the branch 
cut is continued according to the same rule, always separating contour 
lines of constant phase which originated at different points of zero wall 
impedance. 

Each such region, bounded by the 6-axis and the branch cuts (broken 
lines in the diagrams) is now designated by the metallic waveguide mode 
located within it. The normal modes of helix waveguide are then defined 
uniquely, and any further discussions can be made in terms of these 
modes. 

This mode designation in helix waveguide can be defined in fewer 
words as follows: A mode in helix waveguide of finite wall impedance is 
identified with the metallic waveguide mode into which it degenerates 
when the wall impedance phase is kept constant and the wall impedance 
amplitude made zero. 

Modes in any heterogeneous waveguide can correspondingly be iden- 
tified with metallic waveguide modes when the critical parameters are 
subjected to the proper limiting process. All critical parameters should 
be kept constant except that one which in its limit changes the particular 
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heterogeneous waveguide into a metallic waveguide. Requiring the 
procedure to be most direct will in general eliminate any further am- 
biguityv. 

Quite generally it is found that in the y-plane the regions of all TE,; 
and TM,, modes in helix waveguide are unbounded while all TE,, 
modes with n > 1 have a bounded region. Thus, the attenuation of all 
TE,, modes with n > 1 is limited and cannot exceed a certain maxi- 
mum value for any wall impedance. The attenuation constant of any 
of the other modes can be made arbitrarily high simply by choosing 
the proper wall impedance. 

In most helix waveguide applications unwanted mode loss should 
be as high as possible. A more detailed discussion of those modes which 
cannot exceed a certain value of attenuation is therefore in order. A 
typical mode with limited attenuation is TEy . Fig. 4 shows an enlarged 
portion of the y-plane that contains the TE, area. Besides being bounded 
by the 8-axis this area is also bounded by an approximate semicircle as 
branch cut. The maximum loss of aa = 0.0363 for TE, is realized when 
the wall impedance is chosen 


2 | = 0.495 are (Z) = 4.5°, (21) 
| Zo | 
where y lies on the branch cut at the point of highest a. There is, how- 
ever, another y value for this wall impedance on the other side of the 
branch cut, a y value that represents a TM, wave. Its real part is aa = 
0.0350. 

For all practical purposes it does not matter which of these points is 
called TEy and which TM,,. The point with lower attenuation @ is 
therefore the decisive one. To render the attenuation of this point as 
high as possible, it is moved along the branch cut into the saddle point 
at aa = 0.0360. The wall impedance for this condition is 

Z | = 0.487 are (Z) = 4.5". 22 ) 
| Zo | 


At the same time, the other point moves also into the saddle point, 


and both modes degenerate into identity. 

All other modes with limited attenuation behave similarly. 

Using the results for the separation constant k and the propagation 
constant of helix waveguide modes, the coefficient of curvature coupling 
between TE, and unwanted modes was computed for modes with first- 
order (p = 1) azimuthal dependence. For the modes of higher order in 
p the coupling coefficient to TE, in a deformed cross section was com- 
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puted. For p = 2 these coefficients describe coupling in an elliptical 
pipe. For p = 3 it is coupling in a trifoil deformation. 

Of greater practical importance are the coefficients of curvature 
coupling. In Figs. 7 through 12 plots of these coupling-coefficients have 
been made for the modes TE, , TMu , TE» , TMy . Again, the contour 
lines of constant phase and the contour lines of constant amplitude of 
the wall impedance have been plotted as an orthogonal network in the 
plane of complex coupling coefficient c. Some of the lines of constant 
phase run out of the diagrams to very large values of c, indicating that 
the particular coupling coefficient has a pole in their vicinity. Compari- 
son with the propagation constant of the respective modes shows that 
these poles occur at the saddle points of the Z = f(y) plot. Indeed in- 
spection of (8) and 


A 
— [F(7,Z)] 
oY 


from (16) shows that where dF'/dy is zero and Z = f(y) has a saddle 
point the normalization factor NV, has a pole. 

Poles of the coupling coefficients might cause concern; after all, they 
represent very strong coupling to unwanted modes. But since the poles 
coincide with saddle points of Z = f(y) there is always strong coupling 
to the two degenerate modes at the saddle point. Coupling to each one 
of these modes is of opposite sign from the other. The total mode con- 
version stays in quite normal bounds. 

It should be recalled on occasions like this that the normal modes of 
helix waveguide, like modes in any lossy structure, are not orthogonal 
with respect to power. Suppose, for example, that Ao is the amplitude 
normalized with respect to power of a circular electric wave. Then Ao- 
is the power carried by this wave. Let the helix waveguide have a wall 
impedance near (22). Then the two modes TE,» and TMy are nearly 
degenerate with respect to each other. Curvature will cause coupling as 
described by (11). Since the coupling coefficients are very large, even a 
short section of small curvature will generate large amplitudes A, of 
TMy, and A» of TE». One of these amplitudes alone, for example A, , 
would mean seriously high mode conversion. Since TM, and TE, are 
not orthogonal with respect to power, both of the amplitudes A; and A» 
together compensate each other to a small total effect. 

In the plots of Figs. 7 and 9 for TE, and TE, the wall impedance is 
always a single-valued function of the coupling coefficient. In Figs. 8 
and 10 for TM, and TMy», Z = f(c) is multivalued. This observation 
can be generalized to the following statement: Any coupling coefficient 
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Fig. 8 — Coefficient ¢ of curvature coupling between TE»: and TM), in helix 
waveguide of wall impedance Z. Contours of constant magnitude p and phase 
angle ® of Z/Z»o in branches I and II of (cR)-plane; a/v = 4.7 


c between circular electric modes and TM modes in helix waveguide of 
wall impedance Z is a function of Z such that its inversion Z = f(c) is 
a multivalued function. A sufficient condition for this statement is that 
c = g(Z) should have more than one pole, for then each of these poles 


gives a different value Z = f(c) for the same argument c x. Inspec- 
tion of Figs. 2 through 6 shows that the area of every TM mode is ad- 
jacent to more than one saddle point of Z = f(y). As stated earlier, : 


saddle point of Z = f(y) corresponds to a pole of ¢ = g(Z). All TM modes 
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Fig. 9 — Coefficient c of curvature coupling between TE: and TEy2 in helix 
waveguide of wall impedance Z. Contours in (cR)-plane of constant magnitude 
p and phase angle of Z/Zo ; a/X = 4.7. 


therefore have more than one pole of ¢ = g(Z) and Z = f(c) is multi- 


valued. 

Actually the plots of Figs. 7 and 9 for TE, and TEy» might be multi- 
valued too. But when limiting the representation to wall impedance 
values with positive real part, the plots are single-valued. 

To facilitate the representation of the multivalued function Z = f(c) 
for TMy and TMy, branch cuts have been made in the c-plane and 
the different branches of ¢ have been plotted in separate planes. 

The broken lines indicate the border of a particular mode in the e- 
plane. They correspond to the branch cuts of y in Figs. 2 through 6. 
The adjoining modes are always listed in the corresponding area. 


V. APPLICATION 


The results of the numerical evaluations have been applied to several 
problems of helix waveguide design: 





THE BELL SYSTEM TECHNICAL JOURNAL, JANUARY 196] 


Mode Filter 


Sections of helix waveguide are inserted at intervals into plain metallic 
waveguide to absorb unwanted modes. For best absorption of a metallic 
waveguide mode the attenuation of the corresponding helix waveguide 
mode should be as high as possible. The most unwanted mode in metallic 
waveguide is TE, ; it most strongly degrades TE», characteristics 
through mode-conversion effects. A good helix waveguide mode filter 
should therefore have a wall impedance that makes the attenuation 
constant of the corresponding TE. mode a maximum. For the present 
case (a/\ = 4.70) this wall impedance value is given by (22). As high 
as the attenuation is for TE» mode for this design, TE, has quite low 
an attenuation constant 


TE, : aa = 0.0360, 
TEn : aa = 0.00686. 


In metallic waveguide, TE, , although not as objectionable as TE, , 


is still a serious offender. A mode filter should at least represent moder- 




















Re(cR) 


Fig. 10 Coefficient c of curvature coupling between Tho: and TM). in helix 
waveguide of wall impedance Z, contours of constant magnitude p and phase 
angle ® of Z/Z.» in branch I of (cR)-plane; a/A = 4.7 
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ate absorption to TE, . The wall impedance for which the 
TE. attenuation are equal and a maximum is 
Z 


“|= 0.2975 are (Z)' = 12.0°, 
Zo 


and the corresponding attenuation is: 
aa = 0.01158. 


These two wall impedance values are the limits for mode filters. Any 
practical design will be in between. 


5.2 Random Curvature 


Wave propagation in curved helix waveguide is described by gen- 
eralized telegraphist’s equations as coupling between the modes of the 
straight guide. For arbitrary but small coupling these equations can be 
solved approximately. An expression for the added TE loss can be 
written in terms of the coupling coefficients and the coupled mode char- 
acteristics. 

Let the curvature distribution «(z) along the waveguide be a stationary 
random process with covariance 


a(u) = <x(z)x(z + u)>. (24) 


According to Rowe® (see also Ref. 3), the average added TE»; loss can 
then be expressed in terms of the covariance of the coupling coefficient: 


<a> = 
a ] 


L 
2 [ e °°"g(z)(L — z)(P, cos ABnz + Q, sin AB,z) dz, (25) 
n “0 


4 


where L is the length of the line; (¢,R)? = P, + 7Q,, the square of the 
coupling coefficient with c, from (11); and Aa, + jAB, = Yn — Yo, the 
difference in propagation constant of a coupled mode n to the TE 
mode. The summation has to be extended over all coupled modes n. 

For a mere estimate of the effects of random curvature the covariance 
is assumed to be exponential: 

o(z) = <x2>e 7" waa (26) 

where Ly may be regarded as a correlation distance. 

When the correlation distance Lo is small compared to the total length 
L of the waveguide, the average added loss is determined by the rms 
curvature v/ <«?> and Ly : 


2 P.( 2 a A n Lo ) ¢ rAB, Lo 
<a> = «x>Io D, “ t = +s ; 
n AB, Lo" + ( 2nr + Aa, Lo - 





It ‘Sty 


: U Si= 
2 s 


a 
> 








1961 





2°0- 
2 


JANUARY 








JOURNAL, 


gd 
810 
—+—_——__}—_+ 
2°0 


a 
Z 
= 
DL 
Da 
L 


{ 














») 


=f 





OL Bry Jo TT] puv TP soyouvag — Zl “B14 


(Y°)ey (49) 9y 
ZO Ke ’ : 0- 2'0- €0- v'0- 
| | if 




















CONVERSION 


ea) 
~ 
~ 
= 
- 











AND 








MODES 


NORMAL 




















276 THE BELL SYSTEM TECHNICAL JOURNAL, JANUARY 1961 


Equation (27) has been evaluated for a helix waveguide, the TE,» 
attenuation of which is an absolute maximum and for a helix waveguide 
with equal and maximum attenuation for TE, and TEy,. The results 
are plotted in Fig. 13. Also plotted in this figure are the corresponding 
curves for plain metallic waveguide and for helix waveguide with infinite 
wall impedance. The latter design of helix waveguide minimizes TE 
losses in intentional bends. 

Shown in Fig. 13 are curves of the rms radius of curvature as a func- 
tion of correlation distance Lo. This rms value would add 10 per cent 
of the TE ; loss in a perfect copper pipe to the average TE, loss in the 
respective waveguide. 

In calculating the curves of Fig. 13, coupling to the following modes 
of helix waveguide and metallic waveguide has been taken into account: 


TEn, TMn, TEx, TMs, TE:3, TMi. 


Contributions from higher-order modes are small enough to be neglected. 

One important conclusion can be drawn from Fig. 13. When the 
correlation distance of random curvature is small enough — smaller 
than 10 feet in the present case — the added average loss is nearly 


independent of the wall impedance and nearly the same as in plain 


metallic waveguide. This independence is not only true for random 
curvature with exponential covariance but for any random curvature 
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Fig. 13 TE loss in round waveguide with random curvature; a/A = 4.7. 
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with sufficiently flat spectral distribution. The curves of Fig. 13 for 
exponential covariance demonstrate, as a typical example, over what 
range of correlation distance the average added loss is independent of 
the particular jacket structure. 

Random curvature with a correlation distance smaller than 10 feet 
can be classified as a manufacturing imperfection. After all, the in- 
dividual pipe sections which make up the line are usually only 15 feet 
long. Any particular choice of wall impedance therefore does not relieve 
the straightness tolerances which should be met in the manufacturing 
process. 


For correlation distances larger than 10 feet the average added loss 


becomes more and more dependent on the wall impedance. For a speci- 


fied average loss helix waveguide with infinite wall impedance — for 
intentional bends — may be bent most strongly. But even a helix wave- 
guide designed optimally as a mode filter —arg,, = are,, OF arr, 
maximum — may be bent much more than plain metallic waveguide. 

Random curvature with a correlation distance larger than 10 feet 
may be classified as a laying imperfection. Its spectral distribution con- 
tains mainly mechanical frequencies which correspond to sine waves of 
10 feet and more. Such curvature distribution arises from following 
right of ways or the contour of the landscape or just from not installing 
the pipe very carefully. 

The curves in Fig. 13 have been drawn for a specified average loss. 
For very large correlation distance they approach asymptotically a 
constant value. This value corresponds to the normal circular electric 
mode in the particular helix waveguide with constant curvature. Helix 
waveguide for intentional bends, since with Z = it is assumed to be 
lossless, within the limits of the present calculation, may have an 
arbitrarily small radius of curvature. Uniform curvature causes no loss 
in this lossless structure. The curve for metallic waveguide goes to 
infinity. The circular electric mode is not a normal mode of the curved 
metallie guide. 


5.3 Random Ellipticity 


Wave propagation in elliptical helix waveguide is analyzed in a 
similar manner to propagation in curved helix waveguide. 
Instead of (21) the covariance of the cross sectional deformation 


a(u) = ¢<6(z)d6(z + u)> 


is introduced and, for (c,a/6,)? = P, + jQ,, the coupling coefficients 
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Cn = €ni from (13) of a deformed helix waveguide are substituted. Then 
the average added TEm loss is given by (25), and for an exponential 
covariance by (27). 

Equation (27) has been evaluated for elliptical deformations of the 
same waveguides which were analyzed for random curvature before. The 
result is shown in Fig. 14. The rms of elliptical diameter differences 
4(4/ <62>)a, which would add 10 per cent of the TE loss in a perfect 
copper pipe to the average TE» loss in the respective waveguide is 
plotted over the correlation distance Lo. Coupling to all modes which 
are propagating in the metallic waveguide has been taken into account. 
For a/X = 4.70 there are 17 modes of azimuthal order p = 2 propagating. 
Contributions from higher-order modes are small enough to be neglected. 

When the correlation distance is smaller than one foot the average 
loss is independent of the wall impedance. For larger values of correla- 
tion distance the average loss will depend on the wall impedance, but 
this is hardly of any practical significance. Ellipticity is a typical manu- 
facturing imperfection, and will always have a small correlation dis- 
tance. For all practical purposes, cross-sectional tolerances in helix 





HELIX WAVEGUIDE FOR 
INTENTIONAL BENOS 


HELIX WAVEGUIDE 
METALLIC WITH Arte =Are 
WAVEGUIDE 1s 


HELIX WAVEGUIDE 
WITH Ste.,* MAX 


”) 
= 
20 
re) 
25 
a 
Yup 
z 
ae 
Cw 
Le 
_< 
air 
> 
« 
e< 
= 
wo 
20 
<0 
a < 
Ww 
z2 
Us 
cs 
26 
du 
Iw 
ae 
a 
Oo 
of 
= 
x 

















° 
0.01 0.02 90.05 0.1 0.2 0.5 1.0 2 5 10 20 50 100 200 500 1000 
CORRELATION DISTANCE OF EXPONENTIAL COVARIANCE IN FEET 





Fig. 14 — TEo: loss in round waveguide with random ellipticity; a/A = 4.7. 
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waveguide are independent of the wall impedance and the same as in 
metallic waveguide. 

The curves in Fig. 14 have been drawn for a specified average loss. 
For very large correlation distance they approach asymptotically a 
constant value, which corresponds to the normal circular electric mode 
in the particular helix waveguide with uniform ellipticity. 

Metallic waveguide (Z = 0) and helix waveguide (Z = ~) — since 
they are assumed to be lossless — have curves which have a never- 
leveling slope. Uniform ellipticity causes no loss in these lossless struc- 
tures. 


VI. CONCLUSION 


The characteristics of normal modes in helix waveguide can be repre- 
sented as a function of the wall impedance Z. The propagation constant 
y is a multivalued function of the wall impedance, with each value 
corresponding to a normal mode. But for a specified order of azimuthal 


dependence the wall impedance is a single-valued function of the prop- 


agation constant. The most suitable representation of propagation char- 
acteristics of modes in helix waveguide is therefore of contour lines of Z 
in the y-plane. 

Appropriate branch cuts make y a single-valued function of Z and 
lead to a unique mode definition: Any mode of helix waveguide is identi- 
fied by the mode of metallic waveguide into which it degenerates when 
the wall impedance phase is kept constant and its amplitude made 
Zero. 

The attenuation constant of all TE,, modes with n + | is limited. 
The attenuation constant of any other mode in helix waveguide can be 
made arbitrarily high with a proper choice of wall impedance. 

Helix waveguide for mode filters should be designed between two ex- 
treme rules. One makes the TE, attenuation an absolute maximum and 
leads to low TE, loss; the other makes TKy and TH, attenuation equal 
and as high as possible. 

Mode conversion between circular electric and other modes in curved 
or deformed helix waveguide can be calculated from the propagation 
constants and coupling coefficients of the coupled modes. For random 
imperfections the added average TE loss is independent of the wall 
impedance as long as the correlation distance is small. Manufacturing 
tolerances for helix waveguide are therefore independent of the particu- 
lar design. 
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Laying tolerances produce random curvature of large correlation 
distance. They depend strongly on the wall impedance. An infinite wall 
impedance minimizes the average TE loss in helix waveguide curved 


randomly in this manner. 
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Krror-Correcting Codes for 


Multiple-Level Transmission 
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A q-level alphabet is defined as a row vector space over a finite field with 
q elements. The letters of the alphabet are the rows of the vector space, each 
consisting of n symbols from the ground field. The weight of a letter is the 
number of nonzero symbols it contains. The minimum weight of the letters 
of the alphabet, excluding zero, is denoted by d. A relationship is established 
between the alphabet and a set of points S in a finite projective space. There 
is a many-one correspondence between the letters of the alphabet and the 
hyperplanes of the space. The weight of a letter is simply related to the 
incidence of the set S with the corresponding hyperplane. 

Two sets of points in a finite projective space are called equivalent if they 
are related by a collineation of the space. Two alphabets are called equivalent 
if there exists between them, as vector spaces, a weight-preserving semi- 
isomorphism. It is shown that these definitions mean the same thing and 
reduce to the usual definition when q = 2. 

An inequality is established between the dimension of the alphabet and 
the parameters d, q, n. This gives a lower bound for n in terms of the other 
parameters. It is shown that this bound cannot be achieved by alphabets 
with repeated columns. A method is given for constructing a class of alpha- 
bets which attain this bound. It is shown that for the case q = 2 these are 
the only alphabets (in the sense of equivalence) for which the bound is at- 
tained. 


I. INTRODUCTION 


A great deal of work has been done on error-correcting codes for the 
binary channel. In this paper we consider codes for a channel that can 
transmit more than two levels. Multiple-level transmission is practical 
if the channel is sufficiently quiet, as, for example, the submarine voice 
cable. It results in a substantial increase in bit rate and in added flex- 
ibility in choosing a code. One now has four parameters to adjust — the 
number of levels of transmission, the number of information symbols, 
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the number of redundant symbols, and the number of errors it is desir- 
able to detect and/or correct. Of course it cannot be decided without 
detailed analysis whether these advantages will more than compensate 
for the added complexity of the terminal equipment. 

In the binary case, systematic error-correcting codes have certain ad- 
vantages;' in particular, they are amenable to known mathematical tech- 
niques. It has been shown by Slepian’ that the words of a systematic 
code form a group under place-by-place addition mod 2. The natural 
generalization of a group code over the field (0,1) appears to be a vector 
space over a finite field of g elements. We call such vector spaces alpha- 
bets, and their individual elements are called letters. In the general case, 
a “code” becomes an “‘alphabet” and a word (unfortunately!) becomes 
a “letter.”” Each letter is a row of n symbols picked from the ground 
field; the alphabet is a space of row vectors of length n. The q different 
symbols of the ground field correspond to q different transmission levels. 

Because only a restricted type of code is considered, some assumptions 
must be made about the nature of the channel and of the information 
being transmitted. These are as follows: 

(a) The number of transmission levels is a power of a prime number, 
since the number of elements in a finite field is a power of a prime. In 
practice this is not a severe restriction; between one and nine we have 
excluded only the number six. 

(b) The channel is ‘‘symmetric’’ in the sense that every symbol has 
the same chance of getting through correctly, and that the probability 
of one symbol being changed into another is the same for every pair of 
symbols. 

(c) All errors are equally bad. This might be the case, for example, if 
one were ordering merchandise from a mail order house by catalog num- 
ber only. 

With these assumptions the principles of error correction by a q-level 
alphabet are exactly the same as those described by Slepian’ for a group 
code (i.e., a two-level alphabet ). For convenience, the pertinent results 
from Slepian’s paper are summarized in the Appendix. The parameters 
of an alphabet, besides n and gq are 

1. Its dimension as a vector space, denoted by k. The alphabet con- 


A k ° » ° ° 
tains q letters; k is also the number of symbols in each letter which can 


be regarded as carrying information. The remaining n — k symbols are 
added for the purpose of error detection and /or correction. 

2. The minimum weight, d, of the letters of the alphabet other than 
(00 --- 0). (The weight of a letter is the number of nonzero symbols it 
contains.) The quantity d is closely related to the error-correcting prop- 
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erties of the alphabet; if an alphabet is to be capable of correcting all 
occurrences of 1, 2, --+ , e errors in each letter it must have d = 2e + 1. 

The purpose of this paper is to investigate the properties of vector 
spaces over finite fields, particularly those properties which are related 
to the parameter d. The weight of a letter exists only in relation to a 
particular base of the vector space, which is an awkward situation in 
modern algebra. Hence our chief mathematical tool is not algebra but 
finite projective geometry. The connection between binary group codes 
and finite geometries was pointed out by Bose,’ and is easily extended 
to the general case. 

We first establish several new definitions of equivalence between alpha- 


bets. (Two equivalent alphabets have the same error-correcting proper- 


ties.) A lower bound for n is found in terms of k, g and d. Clearly it is 
desirable to have n — k (the number of check symbols) as small as 
possible. It is shown that this lower bound can be attained, but only by 
a restricted class of alphabets. These alphabets are, on the whole, not 
practical for communication purposes unless the expected error rate is 
extremely high. However, the geometric methods used in the construc- 
tion of these alphabets can be applied to find useful alphabets for specific 
cases. The theorems derived for qg-level alphabets apply equally well to 
the case g = 2 and contribute to the theory of binary group codes. 
Il. NOTATION 
In this section we define the notation to be used in this paper and 
introduce Bose’s theorem on the relation between alphabets and 
projective geometries. f 
Let F(q) be a finite field with g elements and characteristic p, and let 
F*(q) denote the nonzero elements of F(q). We consider a vector space 
of dimension n over F(q). Let G,(q) denote the “row space,”’ i.e., that 
particular representation of the vector space consisting of all possible 
n-tuples of elements of F(q). For example, G2(4) consists of the 2-tuples 
(00) (10) (OL) (11) (lw) (1w") 
(wO) (Ow) (ww) (ww) (wl) 
(wO) (Ow?) (ww?) (wil) (ww) 


where w is a primitive cube root of unity. 

Clearly G,(q) has q" members. The qg" — 1 nonzero elements of G,(q) 
can be divided, in many ways, into (q¢q — 1) sets G,, «++ , G4 such 
that G; = AG;, A F*(q). For our purposes it is usually enough to 


t For finite projective geometry, see Carmichael,* Ch. 2; for Galois fields, see 


van der Waerden,® Ch. 5, Seet. 37. 
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examine only one of these sets, for example the first line in the table 
above. 

A subspace of G,(q) is called an alphabet over F(q) and its members 
are called letters. The length of a letter is n and the number of nonzero 
coordinates in a letter is its weight. Every alphabet contains the letter 
(00 --- 0). The minimum weight of its other letters is denoted by d, and 
d is also called the weight of the alphabet. The dimension of the alphabet 
as a vector space over F(q) is k. By @(k,djn) we mean an alphabet @ 
with dimension k, weight d and length (of each letter) n. For example, 
G,(q) is @(n,1,n). 

An alphabet @(k,d,n) contains q‘ letters, from which we pick any k 
independent vectors as generators. We write these as the rows of a 
k X n matrix M(@), the generator matrix of @. For example, 


1 0 1 
eo 2.3 


is the generator matrix of an @(2,2,3). We may assume that no column 
of a generator matrix consists entirely of zeros, for then the alphabet is 
isomorphic to a subspace of G,1(q). 

An ordered set of k elements of F(q), not all zero (for example, a 
column of a generator matrix), may be regarded as the coordinates of a 
point of a projective space 7'.,(q), of projective dimension (k — 1), 
over F(q). We shall adopt the convention that a k-tuple which refers to 
a point of 7.,(q) is to be written as a column vector, e.g., 


| it 


q21 | 
| 


Q = 


ki 


T.1(q) contains (q* — 1)/(q — 1) points; if \ © F*(q), Q and AQ are 
the same point. The points of 7,_;(q) are in one-to-one correspondence 
with one-dimensional subspaces through the origin in G;(q). 

Let us now write the generator matrix of @(k,d,n): 


Q Q -- Q, 
R, qu 
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and call the rows R,, R., --- , R, and the columns Q,, Q2, --- , Qn. 
Regard the columns as a set of points in T,_;(q). There are exactly k 
independent columns, so this set of points spans the space 7'.;(q). Let 
v; be the number of times which some multiple of the column Q; [the 
multiplier being an element of F*(q)| appears in /(@). The correspond- 
ing point in 7,._,(q) shall then have multiplicity »;. We can now intro- 
duce Bose’s theorem. t 


Theorem 1: Let 


be a general point of 7._:(q). Let S denote the set of points Q; , Q» 


2) 


Q,, each counted with proper multiplicity. Then the weight of the letter 


R(A) = MRi + oR + -+- + AR, = E F(q) 


of @ is equal to the number of points of the set S which do not lie on the 
hyperplane 


H(X) = Ayr + Aye + ++ +MY = O 


of Ty 4(q). 
Proof: If, for example, 


Aqui + Aoqa1 I Neus = 0, 

the point Q, lies on H(A). The zeros in the letter R(\) arise from the 
points of S which lie on H(A), and the number of zeros will be the 
number of such points counted with proper multiplicity. The weight of 
R(X) is the number of its nonzero coordinates, which is the number of 
points of S (again counted with proper multiplicity) not lying on H()\). 
This proves the theorem. 

In Fig. 1, the projective plane 7'(2) is over the field (0,1). Note that 


Q:0;Q¢ are also collinear: 


? 


t A different proof of this theorem for the field (01) is given in Ref. 3. 
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Q; 





Q Q 
3 Q. 2 





Fig. | Illustration of Theorem 1. 
Taking points Q, ,Q>,Q3,Q; in Fig. 1 as the set N we obtain a generator 
matrix 
l 
] 
l 


of an alphabet @(3,2,4). It is clear from the figure that there are at 
least two points of N not on any line of T.(2). 


Ill. EQUIVALENT ALPHABETS 


In this section we take up the question of equivalent alphabets, and 


show how Slepian’s definition of equivalence may be extended to the 
more general case. First we discuss what properties one would intuitively 
hope for from such a definition. 

We may consider an alphabet as an array of letters arranged one under 
another in such a way that we can speak of its columns. We know that 
the operations of permuting the columns, multiplying any column by an 
element of F*(q), and interchanging the names of the nonzero symbols 
will not change the error-correcting properties of the alphabet. The defi- 
nition of equivalence between alphabets should allow us to do as many 
of these things as possible. 

From Bose’s theorem we recall that the weight of every letter of an 
alphabet is determined by the properties of a set of points in 7,4(q). 
First we wish that all alphabets derived from the same set of points 
should be equivalent; secondly, if two sets of points S,S’ have, in some 
sense, the same incidence relations with the hyperplanes of 7,_,(q) they 
should give rise to equivalent alphabets. 
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Given a set of points S in 7,_,(q), we derive an alphabet from them 
by means of a generator matrix. We obtain the generator matrix by the 
following steps: 

1. Fix a coordinate system in T,_,(q).t 

2. Write the coordinates of the points of S as columns of a matrix 
repeating each column (not necessarily consecutively) with the proper 
multiplicity. 

The order in which we write the columns is immaterial; also if X, is 
sucha column, we have the option of using AX; , A © F*(q), instead. Thus 
it is apparent that a great many different generator matrices may arise 
from the same set of points. 

We shall presently give separate intrinsic definitions of equivalence 
between two sets of points, two matrices and two alphabets, and show 
how they are interrelated. First we give a brief description of the col- 
lineation group of 74(q).t 

A collineation is a mapping of the set of points of 7;_;(q) onto itself 
which preserves all incidence properties; that is, it sends lines into lines, 
planes into planes, lines through a point into lines through a point, and 


so on. The collineations of T,1(q) form a group, denoted by C(k,q). 


A nonsingular linear projective transformation of coordinates is a 
collineation; so is the (nonlinear) transformation of coordinates induced 
by an automorphism of the ground field F(q). Let P(k,q) be the group 
of linear projective transformations, and A(k,qg) the group of trans- 
formations induced by automorphisms of the ground field. Then any 
collineation of C(k,qg) can be expressed as the product of a member of 
P(k,q) and a member of A(k,q). [Although an element of P(4,q) does 
not in general commute with an element of A(k,q), the two groups 
commute as subgroups of C(k,q).| We recall that an automorphism of a 
finite field of g = p™ elements is always of the form @— 6”", where 6 
is a primitive element; and, fora nontrivial automorphism, 0 < » < m. 
The integers of the field (the elements of the prime subfield) are not 
changed by such a mapping; hence a prime field has no nontrivial auto- 
morphisms, and in this case C(k,p) = P(k,p). 

We now make the following definitions of equivalence: 

Definition 1: The (unordered) sets of points S,S’ are equivalent if 
there exists a collineation of T;:(q) which sends S into S’. We write 
S’ Cis}. 

+ By a fixed coordinate system we mean that the coordinates of every point are 
fixed, except possibly for multiplication by an element of F*(q). In the case of 
finite projective geometries, this involves more than choosing the base points of 


the system. ne é 
t The subject is treated in great detail in Carmichael,‘ pp. 355-372 
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Definition 2: Two (k X n) generator matrices M,M’ over F(q) are 
equivalent if 


M’ = g@M*, M* = MrA. 


Here ® is an automorphism of the ground field applied to the entries in 
M*,g an invertible (k X k) matrix over F(q), ran (n X n) permutation 
matrix, A a (nonsingular) diagonal (n X n) matrix over F*(q). 

Since r has only one nonzero entry in each row and column we can 


always choose A’ so that 
A’x = vA. 


Definition 3: Two alphabets @ and @’ are equivalent if there exists 
between them a weight-preserving semi-isomorphism. 

A semi-isomorphism f between two vector spaces @, @’ is uniquely 
specified by describing what happens to the base vectors R,, --- , R, 
of @, and choosing an automorphism of the ground field. The mapping 


, 


AR) =Ri, ¢=1,---,k, 


k k 
f (> aR.) = +e P( a; Re’ 


1 i=] 


1 


. . , / . . 
is a semi-isomorphism provided that R, , --- , Ry are linearly independ- 


ent; any semi-isomorphism can be described in this way. 

We note also that a weight-preserving mapping of an alphabet @ onto 
an alphabet @’ is necessarily one-to-one; for only letters of zero weight 
in @ can map onto the zero (00 --- 0) of @’. 

In all of these definitions, equivalence has its usual properties; i.e., it 
is symmetric, reflexive and transitive. 

We now show that the three definitions are compatible; that is, in a 


sense to be made precise, 
Definition 1 — Definition 2, 
Definition 2 — Definition 
Definition 3 — Definition 1. 


Theorem 2: If S,S’ are equivalent in the sense of Definition 1, then the 
matrices M,M’, to which they give rise in a fixed coordinate system, are 
equivalent in the sense of Definition 2. 

Proof: Let S be an ordering of the set S, and S’ the ordering of S’ 
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e ° ° ° _— , , yf 
into which S is sent by a collineation g® of T,1(q). If X; ,X; are cor- 
responding points of S,S’ their coordinates are given by 

f \ 


P(x, ) 


Let M(S), M(S’) denote the matrices with columns 


/ 


ee a ere M(S’) = g®M(S). 


Then there exist permutation matrices such that 
M'r’ = M(S’) = g@M(S’), = M(S) = Mrz. 
Hence 
M’ = g@M*, M* = Mra’! = Mr*, 


where m* is a permutation matrix. 
Theorem 3: If the generator matrices 1/,M’ are equivalent in the sense 
of Definition 2, then the alphabets @,@’ derived from them are equiva- 
) 


lent in the sense of Definition 3. 
Proof: We have 
M’ = g?M*, M* = Mr. 
Let @* be the alphabet derived from ./*. We set up a weight-preserving 
isomorphism h between @ and @*, and a weight-preserving semi-iso- 
morphism f between @* and @’. We define h as follows: If R is a letter 
of @ then 
h(R) Raa. 

This is clearly a weight-preserving mapping, since its effect is to permute 
the entries in R and multiply each entry by an eleme®t of F*(q). It is 
also linear, for if R, , --- , Ry are the rows of MW, and R,*, --- , R,.* the 
rows of M* we have 


A(R; ) —_ RjwA mi", 


k k k 
h (> aR.) = > aRiarAv = 2. aR ;*. 


i=] t=1 i=l 


We define f as follows: If R* (m1, 72, °° ,%n) is a letter of @*, then 
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f(R*) = [&(1), P(re), «++ , &(r,)]; f is weight-preserving, since ®(r) = 0 
implies r = 0. 
To show that f is a semi-isomorphism, 
* f , 
a Gs 
l a . . ; 
we observe that g M’ is also a generator matrix of @’. Let R, , 
7 I : ; , , , 

be the rows of g M’, and let R, = (ri,ric, «++, Trin). Let Ri*, ---, 
? * * * 


» * Ba l , 
be the rows of M*, with (ri ,*ri2,* -- >, Tin’). Sinceg M 


we have 


[ ran ) P/ ro joes , Pi ley * )|, 


{(R,*) = R’ : 


Then 


k k k 
f (> aR *) = E (> ar *) ,® (> aT i2* 
l 1 l 


Since ® is a field automorphism this becomes 


(Ean) 


k k k 
| Pua P(rii*), Dy Plas )P(ri2*), +++, D) ba) P(r | 


1 l i=] 


k 


, 
Ve Pla; )R 


1 


We then have 
R; ot R;* = R’ ’ La.R, mn za:R;* : » TP(a RP’ 

and hf is a weight-preserving semi-isomorphism between @ and @’. 

Theorem 4: Let @,@’ be equivalent alphabets in the sense of Definition 
3, and M,M’ be any generator matrices of @,@’. Fix the coordinate sys- 
tem in 7_,(q), and let S,S’ be the sets of points whose coordinates are 
the columns of Wand MW’. Then S and S’ are equivalent in the sense of 
Definition 1. 

Lemma: Let the alphabets @,@* be related by a weight-preserving 
isomorphism w; 1/,\/* are generator matrices of @ and @* such that 
M* = w(M). Then in any coordinate system in 7,_;(q) the columns of 


M and M* give rise to the same (unordered ) set of points S 
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Proof of Lemma: If R,, --- , Ry; Ri*, --- , Ry* are the rows of M 
and M* we have 


k k 

w(R;) = R,*, w( aR) = )) aR,*. 
i=] t=] 
Let (jy +++ yx) be the coordinates of the general point of 7,:(q). Map 
the letters of @ onto the hyperplanes of 7;_:(q) as follows: R; maps onto 
Yi (0, Zz aR; maps onto > iY, 0. Because of the isomorphism be- 
tween @ and @* we have a similar mapping of the letters of @* onto 
the hyperplanes of 7,~:(q): R,* maps onto y, 0, >> aiR* maps 
onto >, Yi Q. 

Let / (6;;) be the incidence matrix of points and hyperplanes in 
T.1(q), Where 6;; = 1 if the 7th point lies on the jth hyperplane and is 
zero otherwise. Each row (column) of J contains (g° ' — 1)/(q — 1) 
ones and q‘ ' zeros. The matrix / for the projective plane 72(2) is il- 
lustrated in Table I. 

The matrix / is nonsingular. This is easily seen by considering the 
product /-J. In this, all terms on the main diagonal are equal to the 
*“" _ 1)/(q — 1), on a hyperplane. All other 
terms are equal to the number of points, b = (q° ” — 1)/(q — 1), on 


number of points, a = (q 
the intersection of two hyperplanes. The determinant of the matrix is 
then 


la + (u — 1)b\(a — db)". 


When we substitute the values for a,b, the first factor becomes 


q’ l ae 1 F 
q- l . 
hence the determinant is not zero. (We assume k > 1.) 


Let P,, P2, ++: , Pu, uw = (q¢' — 1)/(q — 1), be the ordering of the 
points of 7,1(q) as they appear as columns of /. Let S,S* be the sets 


TABLE | - INCIDENCE MATRIX FoR PoINTs AND LINES IN T'(2) 


100 010 001 110 101 oll 111 


0 0 0 
l 0 l 
| l 0 
0 I 
0 0 
1 0 
0 1 


hiuud va 
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of points whose coordinates are columns of /,M™* respectively. Assign 
to P; the multiplicity n;(n;*) with which it appears in the set S(S*). 
If P; does not appear in S(S*), nj = 0 (n;* = 0). Form the column 


vectors 


The 7th term of the matrix product /n is the sum of the multiplicities 
of the points of S which lie on the 7th hyperplane. By Theorem 1, this 
is the number of zeros in the corresponding letters of @. 

Since the isomorphism between @ and @* is weight-preserving we have 

In = [n*, 
or, since J is invertible, 
n= n*. 
Hence the set of points S* is at most a rearrangement of the set S. 

Proof of Theorem 4: @ and @’ are related by a weight-preserving semi- 

isomorphism f. Let R,, --- , Ry be the rows of the generator matrix M 


of @. RY = f( Ri), ---, Re = f(R,) are k linearly independent letters 
of @’, which we may take as the rows of a generator matrix M” of @’. 


We can describe f as follows: 


k k 
f(R;) = Ri, s(d «R,) = > (a;)Ri, 
i=] i=] 
where ® is an automorphism of the ground field which is uniquely deter- 
mined by f once we have chosen M. 

Let R* = @ (Ri), i = 1, ---, ky Ri*, «++ , Ry* are linearly inde- 
pendent. Let ./* be the generator matrix formed of these rows and @* 
the alphabet derived from M*. The mapping h of @’ onto @* induced 
by ® ' is clearly a weight-preserving semi-isomorphism. 

Consider the mapping fh between @ and @*. We have 


<> 


h 


re 
. f - ” ” 2 
La:R; > Lb(a;)R; » & [2b(a;)R;] = Ta:R;*. 


Since f is weight-preserving by hypothesis, fh is a weight-preserving 
isomorphism between @ and @*; M and M* are corresponding generator 
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matrices under fh, hence by the Lemma they arise from the same set of 
points S in Ty1(q). 
Let S” be the points of 7;.1(q) corresponding to the columns of M”. 
” 
z) [x1 
x! 71, X* =]: 


” 
\LE / Xk 


*\ 


* 


are the 7th columns of /” and M* respectively, we have 


” 
[x | @(7,*) 


ry P(2;,.*) 
Hence the set S” is obtained from the set S* by a collineation C, of 
T pst q). 

Let M’ be any generator matrix of @’; then M/’ = gM”. Let S’ be the 
points of 7..(q) corresponding to the columns of M’. S’ arises from S” 
by a linear projective transformation, i.e., by a collineation (2 . 

We have then 


Ss’ = C's ( Ss” ) = CoC S a 


which proves the theorem. 

It can be shown from Theorems 2, 3 and 4 that a complete equivalence 
class of sets of points gives rise to a complete equivalence class of 
matrices; a complete equivalence class of matrices gives rise to a com- 
plete equivalence class of alphabets; and this in turn gives rise to a 
complete equivalence class of sets of points. The details of these cor- 
respondences are quite complicated, since an unordered set of points can 
give rise to many matrices, and different generator matrices can produce 
the same alphabet. 

Theorems 2, 3 and 4 are, of course, true over the field (0,1). We rewrite 
our definitions for this field, since they take a simpler form. ® is the 
identity, and the only possible choice for A is the unit matrix. 

Definition 1’: Two sets of points S,S’ in 7,_;(2) are equivalent if 
they are related by a linear projective transformation of coordinates. 

Definition 2’: Two (k X n) matrices M,M’ over F(2) are equivalent if 


M = gM’n, 


where x is an (mn X n) permutation matrix, and g an invertible (k X k) 
matrix over F(2). 
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Definition 3’: Two alphabets @,@’ over F(2) are equivalent if they 
are isomorphic as groups in such a way that corresponding elements 
have the same weight. 

It will be recognized that this is, in fact, the familiar definition of 
equivalence for alphabets over (0,1). 


IV. RELATIONS BETWEEN k, d, n 

In this section we establish certain relations between the parameters 
k,d,n, which are necessary conditions for the existence of an alphabet 
@(k,d,n). We assume, as before, that the alphabet has no column con- 
sisting entirely of zeros. 

Define Z[{x] to mean the least integer greater than or equal to the 
rational number z. 

Theorem 5:+ A necessary condition for the existence of @(k,d,n) is that 


7| 1 fd -1 
“|e (T=1) 4] 


Proof: As before, let J be the incidence matrix of points and hyper- 
planes in 7'.1(q). 
Let J be the complement of / obtained by replacing zeros by ones and 


‘ ° . k—1 
ones by zeros. J is symmetric; each row (column) contains q ~ ones and 


Ll+qt+-:- + ¢ ° zeros. 

The matrix J for the projective plane 7.(2) over the field (0,1) is 
illustrated in Table IT. 

Let 


‘and n, stands for the multiplicity of the 


where yu 1+qct 
point P; of Ty 4(q). 

Consider the expression Jn. The product of the 7th row of J with 
the column of n; is the sum of the multiplicities of the points P; which 
do not lie on the 7th hyperplane. By Bose’s theorem, this is the weight 
of the letters of the alphabet corresponding to the 7th hyperplane. Now 

+ This theorem has been obtained for the field (0,1) by many authors in as many 


ways. See for example, Ref. 6, Theorem 5; Ref. 3, Eq. (52), and other authors 
quoted in Ref. 3 
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TABLE II —J = 


CODES 
= COMPLEMENT OF / 


010 001 110 101 
0 0 
l 0 


define a column vector 


d 


inequalities 


Since our alphabet is assumed to have minimum weight d, we have the 


Jn = d. 


Since we may assume d = 1, these inequalities imply that there must be 
at least one point of nonzero multiplicity not lying on any given hyper- 
plane — that is, the points of nonzero multiplicity span the space 
T-1(q). 


Hence, given k and d, the least value of n for which there exists an 
alphabet @(k,d,n) is the minimum value of 


where n; , 7 


2%; 
=1 
eee 


, are nonnegative integers which satisfy /n 2 d 
By adding all the inequalities of /n = d, we obtain 
A 5 | 


k—1 
» n; 
i=1 


a (i + @ + ++ + qi ad 
or, setting 
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k—1 . 
In the case that d = q’ , for any value of k the lower bound becomes 


=l+qgt+---+q". 

In this case the lower bound is the largest possible lower bound, as it is 
achieved by the alphabet which corresponds to ny = no = ++: = ny = 1, 
that is, the alphabet which results from taking every point of 7; 1(q) 
with multiplicity one. 

One has an intuitive feeling that alphabets with the least n for a given 
k,d are likely to have no repeated columns if this is possible. This is 
partly justified by the following theorem. 

Theorem 6: If a generator matrix of @(k,d,n) contains a repeated 
column [in the sense that Q,; = AQ, for some A of F*(q)], then 


l gq? = 
nzz| (! Ja] +2. 
” 1a g-] 


For the purposes of this proof and the succeeding lemma we write the 
above inequality as 


n= z| : (1 can a] + 2. 
q _— l q' 


Proof: Let P be the point of 7,:(q) which corresponds to the re- 
peated column. Choose a coordinate system in which P is one of the 
base points, say P = e,. We then have an equivalent alphabet @’ 
which may be written 


The letters of @’ to which the first two columns contribute zeros form 
a vector space @; @ is generated by the rows 2, --- , k of M(@’). The 


minimum weight of the letters of @ is at least as great as the minimum 
weight of the letters of @’. Hence the alphabet @ has parameters k — 1, 
d',n — 2, with d’ 2 d. By Theorem 3 we get 
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It is clear that, if d < 2q*" 


I 


We need a little more, namely: 
k—1 
Lemma: If d S q’, then 


i (-p)eree 


Hence 


wre 

styl i 
all 1-2 d+ 

~q-!1 q* 


Theorem 7: lfd S q’ ‘the bound given in Theorem 3 cannot be at- 
tained by an alphs in with repeated columns. 

This result is not surprising in view of the remark at the end of the 
proof of Theorem 3. If d > gq‘ ', the inequality of Theorem 5 gives 
n> (q — 1)/(q — 1); i.e., n is larger than the total number of points 
in the space 7',,(q). Thus we must have repeated columns in the 
generator matrix. 

By repeated applications of the procedure of Theorem 6 we can write 
down lower bounds for the n of alphabets having a given number of 
columns with given multiplicities. However, this does not seem very 
interesting; we will first say what we can about alphabets with no re- 
peated columns. We assume from now on that we are dealing with such 
alphabets. 


V. A CLASS OF ALPHABETS 


In this section we describe a class of alphabets for which the bound of 
Theorem 5 is attained, and show how other alphabets which attain this 
bound may be derived from them. 
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We can immediately write down the class of alphabets.t Choose a 
fixed k, and consider the following sets of points in 7;.,(q): 
(0) The set Sp of all points of 7T4~1(q): 


No l + q + “ee + q 2 do = 


Every letter of this alphabet has weight do 
(1) The set S, of all points but one of T,1(q): 


m=qtqt+-::-+¢'; q 
The (1 + q¢ + --- + q ~) hyperplanes through the omitted point 
correspond to letters of weight g‘', other hyperplanes to letters of 
weight g° ' — 1. 
(2) — Bs all points of 7,_1(q) except for the (1 + q) points of a 
line Ly. 


The (1 + q+ --- + q° *) hyperplanes through 1, correspond to letters 
of weight q ' others to letters of weight q — q. 

(3) —S; = all points of 7,_,(q) except for the (1 + g + q°) points 
of a plane P2. 


q 


The (1 +q+--- +4 *) hyperplanes through P2 correspond to letters 
of weight q° ', other hyperplanes to letters of weight g° ' — ¢. 


(k — 1) S; all points of 7. 1(q) except for the points of a 
hyperplane 


The omitted hyperpl: ane corresponds to letters of weight gq“ ', all others 


to letters of weight g° | — q/ 
It is easy to verify that for — alphabets the bound of Theorem 3 is 
attained. Consider 


t For the case of g = 2 some, or all, of these alphabets have been found by other 
authors by different methods. See, for example, Refs. 3 and 6. They are, of course, 
picked up by any systematic search, such as linear programming. @,_: is the Reed- 
Muller code for m = n, r = 1 
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l q' — ] i 1 ko 1 q' —] k—1 i-1)\ 
eo ha ie ie 

k u k 
qd ‘) — . 1 (: ‘) (q‘ "— 1)q' ' 

q-1 g\q-1 
“-1/; ¢-1 
. (« qe ) 


Since g 2 2, this quantity is less than one; i.e 


“9 


P 
l>n-- (’ ‘) a > 0. 
q I q- l 


It will appear presently that for q = 2 these are the only alphabets 
which attain the bound of Theorem 5. The case g > 2 is more compli- 
cated. 


Suppose that @(k,d,n) is an alphabet (with no repeated columns) for 


OF ns l 
‘= z| d yal. 
q- | 


which 


Write 


Qd = sq’ ; = 9 : Oo < 3s. ¢ 


where r and s are integers; s cannot be zero since d is positive; s S 
: k—lorp 
sinced Sq. Then 


n=Z | « +t~ . | =d+s. 
q 


If we remove 7 columns from the generator matrix of @ in such a way 
that the remaining matrix is of rank k (this is always possible for 
n < n — k), we obtain an alphabet of length » — » and minimum 
weight d = d — 7. Let us consider the worst case, i.e.,d = d — 7. 
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Lemma: If 
then 
provided that 


Proof: 


_ Q 
q 1 


, ¢ 
«Side ~~) ~ St 
q* l 
This is equal tod + s — nif and only if (Qn + r)/q°' < l;ice., 
oe i SS ibe dy e —(q¢g-— 1) 
n Q i d qi , 1 qi 


- 1) ( 1) ira 
q— —(q- ; 
nw \q d gi — ] 


Now suppose that d > d — », and 7 satisfies (2): 


Z| (1 + :) a| (1 +f g) (d — n)| a = >. 
q q 


By Theorem 3 applied to the alphabet @(k,djn — 7), we have 


n—-n2= (1 + @ al. 
q 


Hence only equality is possible. 


Theorem 8: If Q@(k,dn) attains the bound of Theorem 5 and 


@(k,dyn — n) is obtained from it by removing 7 columns from a gen- 
erator matrix of @, where 7 satisfies (2) in such a way that the remaining 
matrix is of rank k, then the new alphabet also attains the bound of 
Theorem 5. 
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We remark that if we select the columns with proper care, it is possible 
to remove more than the number given by (2) and still obtain an alpha- 
bet which attains the bound of Theorem 5. The alphabets @,, --- , @._1 
listed at the beginning of this section are examples. 

We now reformulate (2) in a more convenient form. We observe, 
from (1), that, since d is an integer, so is (sq" ' — r)/Q. Subtract from 
it the integer 


s(q‘ — 


s(q— 1) — - 
S\q Q 


and we find that 


ge - 


ge+:-»-+q4+l1 


is also an integer. We have two cases: 
.s=Q,r= nV lu S q — 1 from (1)}. 
Then (2) becomes 


l 
<(q-—1)- 
” q ut G 


or, since all these symbols represent integers, 


2S(¢gq-1)—4p. (3) 


is << Q,r=uwQ+s8 E < (1 —— = \ — 1) from (1 |. 
q'?- i! 


Then (2) becomes 


In case i we have, from (1), 
k-1 
d=4q — yp, Osu 


The alphabet @o corresponds to the case u = 0, and the alphabet @, to 
uw = 1. From the alphabet @ we can subtract any number ny S q — | 
of columns and obtain an alphabet which attains the bound of Theorem 
5. (The alphabet @, is obtained by subtracting one arbitrary column. ) 
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In case ii we have, from (1), 


a= _Wts 


=s(q— 1) —» 
Q Q _ " 


For the alphabets @», --- , @,, we have u 0. This is readily verified 
by direct calculation: 

= q AEs 1) (q—1)q' ‘(q -->> t+ q-+1). 
Thus 

s rial -++ + q'"), p = OQ. 
We shall show that these are the only alphabets besides @» for which 
a= 6 

The generator matrix of an alphabet @(k,d,n) with no repeated col- 
umns consists of a subset of the columns of the generator matrix M(@p). 
Let S be the generating points of @(k,d,n) in T,_,(q), and denote by 
C(.S) the points of 7,,(q) which are not in S. 

Let v be the number of points in C(.S) and 6 the maximum number of 
points of C(\S) which do not lie on a hyperplane of 7._,(q). The alphabet 
@ then has length ny — v and weight dy) — 6, where ny (q’ — 1) 
(gq — 1), dy) = ¢' ‘ are the parameters of @. Using Theorem 5 on these 
numbers, we obtain 


v\ q 
Since »(q — 1) is an integer we may replace this by 
v(q L) =e — a. 
This is the best we can do, since 6 S q 
By some further manipulation we find that for the alphabet a, gen- 


erated by @ — C(S), to attain the bound of Theorem 5 we must have 


(46 ~ }) > Ae - 2) S vl¢ — 


Ll) So — 1. (6) 


We also wish to have an alphabet with » = 0; for such an alphabet 


k—1 
Sq — § 
/ = 2 _ l r 
( Q q 


dy : Ft _ 3(q — 1), 


k 


\q 
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i.e., gs — 1 is divisible by (q — 1). From (6), the only possibility is 
v(q—1) = g — 1. (7) 


We also observe from (6) that if gq = 2 we have »(q — 1) = gé — 1 
without any other considerations. 

To justify our statement that @., --- , @,. are the only alphabets 
besides @y for which » = 0, we prove the following theorem. 

Theorem 9: If C(S) is a set of v points in T,_;(q), with 6 defined as 
above, and »v(q — 1) = qgé — 1, then C(S) is the set of all points of a 
linear space in 7;_,(q). This, of course, implies that 


p= l+et: +9, t= ¢, Isssk-2. 


Conversely, if C(.S) is the set of all points of a linear space, then the 
alphabet @ has » = 0 and attains the bound of Theorem 5. This we have 
already verified. 

Proof: We have v — 6 = (v — 1)/q, sothat (v — 1) must bea multiple 
of g. If vy = 1 the corresponding alphabet is @, , for which » = 1. 

The proof is by induction on 6; we start by proving the theorem for the 
case (vy — 1) q 1: i.e., 6 q, v q + |. 

Lemma: If v 1 + qandé = gq, the (1 + q) points XN), X,, --- , X, 
are collinear, however large the containing space. 

An equivalent statement, which is the one we prove, is: If every 
hyperplane of 7'..(q) contains at least one of the points Yo, X,, --- 
X,, then Xo), X,, ---: , X, are the points of a line. 

We may assume that there is one hyperplane, say Y; = 0, which con- 
tains exactly one point X;, which we may call X,. Pick another point 
for Xo and let the coordinates of these two points be e; , es. We assume 
the coordinate system normalized so that the first nonzero coordinate of 


every point is unity. Write the coordinates of the X; as columns of a 


matrix as follows: 
X) X 
0) 
l 
0) 
S. 2 neiael 


By Theorem 1, every letter of the form aY; + 8Y2 must contain at least 
one zero coordinate. For a 0 (8 0) we always have a zero in the 
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second (first) place of such a letter. In the (q¢ — 1) letters of the form 
aY,+ Y2,a © F*(q), the zero mustoccur in one of the places 2, 3, --- , q. 
Hence the a2, a3, -+: , ag above must denote some arrangement of all 
the elements of F*(q). 

Consider now letters of the form 


ay, + BY» + vY3. 


Again, the first two coordinate places take care of those letters for which 
one of a,8,y is zero. Hence we restrict ourselves to letters 


aY,+ BY.+ Y3, a,B ¢ F*(q). 


Each such letter must have a zero in one of the places 2, 3, «++ , @. 
»g 


We note that there are (q — 1)° such letters, and (q — 1) coordinate 


places. 

Suppose now that b. ¥ 0. We shall count the number of letters to which 
the X. column contributes a zero. We may choose any a in F*(q) such 
that a ~ b.. B( 0) is then uniquely determined by the equation 
[in F(q)] 


Bay —(a + be). 


Hence if b. ¥ 0 the X, column contributes a zero to only (q — 2) letters. 
If bp = 0 we have (g — 1) choices for a, and 8 is determined by 


In this case the X, column contributes a zero to (q — 1) letters. 

Hence the only possible choice for b; is b; = O for all 7. 

The same argument shows that all rows Y;,7 > 3, consist entirely of 
zeros. The coordinates of X., --- , X, are linearly dependent on those 
of Xo,X,; that is, the points X., ---, X, all lie on the line joining 
Xo,X1. 

Returning now to the main theorem we make the following induction 
hypothesis: 

Let C(\S) be a set of v points in 7,_,(q), with 6 defined as before, and 
such that 


(q — 1)» = gb — 1. (7) 


gq <6 Sq’ ', and assume that Theorem 9 is true for values of 
q’ . We wish to prove that 
v=l+qtqt::-+q". 
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From (7), v — 6 = (v — 1)/q = A, where h is an integer greater than 
1. h = 1 is the case already considered in the Lemma. Also 


6=v—h=hat+1-—Ah. 


An arbitrary space of dimension (k — 3), say Dy, in Ty4(q) will 
contain a number a@ of points of C(S). We wish to find a lower bound 
& for a. 

There are (¢q + 1) hyperplanes of 7;_1(q¢) which pass through D,_; . 
Denote by Bo, 8:, +: ,8, the number of points of C(S), outside of 
D,_3, contained by these hyperplanes. The hyperplanes through D,_,; 
contain among them all points of 7._;(q), so certainly all of C(S). We 
have then 


a+ > 8; = vy=hq+1, 


at+B2v—6=h. 


A lower bound for @ is obtained by making all the 8; equal, 8, 
and replacing ‘*2” by ‘““=” in (8). Then, 
a+ B = h, 
a+ te + 1)B = hq + 1. 


Solving these equations, 


aed § 


q @ 
Let a’ be the least integer containing &. We note that a’ > 0. 

We may assume that some hyperplane, say Hy_2 , of T,1(q) contains 
exactly » — 6 = h points of C(\S). Call this set of points C(.S’). Each 
hyperplane of H,_2 is a (k — 3)-dimensional subspace of T._:(q), and so 
by the previous result it contains at least a’ points of C(S’). 

lor CCS’) we have 
h-1 


q 


/ 


oshkh-a Ssk- 


Therefore, 


qo — 1s qh-—h+1—1 = (q— 1D)h, 
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v(q-—1) 2 @ — 1. 
Comparing this with (5), only equality is possible; i.e., 
go — 1 = (¢ — IDA. 
This implies that (h — 1)/q is an integer, and 


h-1 
q 


sé =h— 


((S’) is thus a set of points with 


c 


and (q— 1) = q6 — 1. 


q 
Since q’ ° < 6/q “ we can apply the induction hypothesis, which 


gives us 


l+qte:-4+q 


, r—I 

y qv +1 LP o> =F @ 

and the points C(.S’) are all the points of a linear space B,» in Hy_». 
We can always find in Hy,» a (k — 3)-dimensional subspace, say 

D,_3 , which intersects B,_» in a space of dimension (r — 3), and thus 

contains exactly 


l+¢er 
points of C(S). 
Consider the hyperplanes of 7.;(q) which pass through D,_; . From 
(8), we have for these 
h—-1 
q 


8, =B=h- 


so that the total number of points of C(.S) in each hyperplane is 
a& + B = A. 


By the previous argument the intersection of C(S) with each hyper- 
plane is a linear space of dimension (r — 2). These spaces have in com- 
mon a linear space of dimension (r — 3), the intersection of B,» and 
D,_;. Hence the set of all their points is a linear space of dimension 
(r — 1). This proves the theorem. 
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We will now summarize the results of the last section. For g = 2, the 
alphabets @p , @:, +--+ ,@,—1 introduced at the beginning of the section 
are the only alphabets which attain the bound of Theorem 5. For g > 2 
these alphabets attain this bound, and have the further property that 
any k-dimensional alphabet obtained from them by removing up to 
(q — 2) arbitrary columns of the generator matrix (q¢ — 1 for @») also 
attains this bound. They are the only alphabets with this property. 
Clearly the alphabets @p , @:, --- ,@:1 are completely determined, up 
to equivalence, by the values of the parameters k,d,n. For a given k, 
d and n are restricted to a certain set of values defined at the beginning 
of this section. 
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APPENDIX 


Slepian’s Error-Correction Procedure 


Let F(q) denote a finite field, and G,(q¢) the group, of order qg", of 
all possible rows of n symbols picked from F(q). The group operation 
is place-by-place addition under the rules prevailing in F'(q). Let A be 
a subgroup of G, . [For the present purposes A need not be a vector 
space over F'(q); the two concepts are the same if and only if F(q) is 
a prime field.] 

Partition G, into cosets with respect to A, with an element of least 
weight in each coset. being picked as ‘‘coset leader.’’ The element (00 
-++ ()) is, of course, the coset leader of A itself. The cosets are formed 
into a table as illustrated in Table III. The group A is the first row of 
the coset table. The first column of the table contains the coset leaders. 
In the case of Table III these are, besides (0000), all the elements of 
weight | in G3(3). 

The element in the sth row and the /th column of the coset table is 
obtained by adding the sth coset leader to the element (of A) in the 
first row and the ¢th column. The sth row is exactly the coset deter- 
mined by the sth coset leader, and every element of G,(q) appears ex- 
actly once in the table. 
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TasLeE II] —Cosets with Respect to A For G,(q) 


2 3 4 5 6 7 


0000 1011 0112 1120 1202 | 0221 
1000 | 2011 1112 | 2120 | 2202 | 1221 
2000 0011 2112 | 0120 | 0202 | 2221 
0100 1111 | 0212 1220 | 1002 | 2122 | 0021 
0200 1211 | 0012 1020 1102 0121 
0010 | 1021 | 0122 1100 | 1212 0201 
0020 1001 | 0102 1110 1222 | 0211 
0001 1012 | 0110 1121 1200 0222 
0002 1010 | O111 1122 1201 0220 


The error-correction procedure is as follows: If the received element 
is a letter of A it is accepted as correct. If not, it is located in the coset 
table, say in row s, column ¢, and the letter of A in row 1, column ¢ is 
substituted. 

It is clear that the example of Table III will correct all single errors. 
Column 2 contains, besides (1011) which belongs to A, all the elements 
of G3(3) which differ from (1011) in exactly one place. 

In general, if it is required to correct all single, double, etc., errors it 
is necessary that all elements of G,(q) of weights 1, 2, ete., appear as 
coset leaders in the coset table formed by A. Let d be the minimum 
weight of the letters of A, other than zero. The coset formed by a leader 
of weight 1 will consist of elements of weight at least (d — 1). Hence 
all elements of G,(q) of weight 1 appear as coset leaders if and only if 
d = 3. Similarly, all elements of weight 2 appear as coset leaders if 
and only if d 2 5. If it is required to correct all e-fold errors, the al- 
phabet A must have d 2 2e + 1. 
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Short-Term Memory in Vision 


By E. AVERBACH and A. S. CORTELL 


(Manuscript received August 8, 1960) 


Experiments are performed that demonstrate some of the functional prop- 
erties of short-term storage in the visual system, its decay, readout and 
erasure. Results indicate that the visual process involves a buffer storage 
which includes an erasure mechanism that is local in character and tends to 
erase stored information when new information is put in. Storage time ap- 
pears to be of the order of one-quarter second; storage capacity is more diffi- 
cult to assess. 


I. INTRODUCTION 


There can be little doubt that eye movements play an important role 
in the perception of form, and that perceptions of complicated visual 
fields are built up from information gathered during many fixations of 
the eyes. But eye movements over a complicated visual field are unpre- 
dictable from subject to subject and from time to time with the same 
subject. They may therefore be an annoying source of variability in 
perceptual experiments, and experimenters frequently find it desirable 
to eliminate them. This is usually done by using a tachistoscope, a de- 
vice for presenting brief exposures of visual material. The position of 
the eyes is kept fixed at the crucial time by having the subject fix his 
eyes before exposure of the material, and by using exposures sufficiently 
brief that the subject cannot change his fixation during the exposure. 
To accomplish this, exposure times must be less than the reaction time 
of the eye for a change of fixation (150-200 milliseconds). Actual meas- 
urement in a tachistoscopic situation has shown that exposures of 100 
milliseconds or shorter satisfy this purpose.! 

At first thought, it may seem unreasonable to study visual perception 
under the peculiar condition of tachistoscopic experiments. Some ques- 
tion might be raised about whether data obtained in this way can be 
generalized to natural perceptual situations. It can be argued, however, 
that in a very real sense the tachistoscopic situation is not an unnatural 
one. For it is well established that, in scanning pictorial or printed mate- 
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rial, the eyes do not take in information continuously (Ref. 2, p. 493). 
They fixate first on one point and then move rapidly to another. The 
fixations are relatively long, but the movements between fixations are 
so quick that they smear the image drastically during the motion. Thus, 
normal vision involves the processing of discrete exposures very much 
like those presented in a tachistoscope. It has been shown, in fact, that 
reading performance is better if the necessity of moving the eyes is elim- 
inated by presenting reading material serially by means of a tachisto- 
scope.® 

To anyone who has ever seen objects illuminated briefly by a spark 
or other kind of brief flash it is evident that the visual impression of the 
illumination lasts longer than the flash. Even a millisecond flash seems 
to last a noticeably long time. Because of this persistence, writers on 
perception, particularly tachistoscopic perception, have assumed the 
existence of some kind of short-term storage in the visual system. This 
is implied in their use of such terms as positive afterimage, retinal per- 
sistence, persistence of vision, etc., in interpreting tachistoscopic per- 
formance. But little work has been done to characterize the functional 
properties of the storage, its decay, readout and erasure. In this paper 
we will discuss some of the older studies that bear on these matters and 
report a few experiments that demonstrate some properties of this vis- 
ual short-term storage. 


Il. MEMORY EXPERIMENT 


A very old tachistoscopic experiment is the span-of-perception ex- 
periment. Its aim is to determine the maximum number of objects a 
person can take in at a glance, the objects being dots, letters, digits, 
words, etc. Typically, the experimenter makes up cards having different 
numbers of items on them. Starting with cards having one item, he 
keeps increasing the number of items presented until the subject begins 
to make errors. The perceptual span may be taken to be the maximum 
number of items that the subject can report perfectly. More usually, 
the criterion used is the number of objects reported correctly 50 per 
cent above chance. 

Spans of perception measured by different investigators are surpris- 
ingly consistent considering the wide range of conditions under which 
these spans have been measured. The span for letters or words‘: is 43 to 
5 and for dots®:* about 8. What limits the span of perception? Of course, 
anything that affects legibility — brightness, contrast, sharpness, etc. 

will under some conditions affect the span. But once reasonable legi- 


bility is obtained, increasing the bnghtness and contrast and sharpness 
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does not improve performance. Under conditions of good legibility the 
limitation is elsewhere. 

Two possibilities suggest themselves. First, the span may be limited 
by the capacity of the visual storage. It may be that, as the number of 
items put into the storage is increased, resolution of the individual items 
is destroyed. The other possibility is that resolution of the storage is 
perfectly adequate for numbers of items of the order used in span of 
perception experiments, but that the storage time is too short; i.e., the 
subject does not have enough time to read more than a few items into 
his more permanent memory before the decay of the short-term storage. 

Selecting between these alternatives presents something of a problem. 
How does the experimenter determine how many items a subject can 
store visually if the subject, asshown by span of perception experiments, 
can report correctly only a limited number (4 or 5) items? This difficulty 
was circumvented by Sperling’ who, instead of requiring that his sub- 
jects report on the whole of a complicated tachistoscopic presentation, 
had them report on only a part. He exposed briefly three rows of four 
randomly chosen letters each. Then, after a variable delay, he presented 
a tone signal of either high, middle or low pitch which indicated to the 
subject that he was to report on the upper, middle or lower row of let- 
ters — whichever was indicated by the tone. Since the subject was not 
familiar with the arrays of letters and was not given the instruction tone 
until the visual stimulus was turned off, he had, in effect, to store the 
whole array. By this method Sperling was able to show that subjects 
can store as many as 9 letters of a 12-letter array —- and even more 
when arrays having more letters are used. 

The experiment to be described, although conceived independently of 
Sperling, is of essentially the same form as his. The essential difference 
lies in the use of a visual signal to designate the part of the array to be 
reported by the subject. This has the virtue that it assures that the 
array and signal are transmitted to wherever they are processed in the 
brain at approximately the same rate. It is known that the reaction 
time to a light is significantly longer than the reaction time to a tone 
(Ref. 2, p. 16). 

A 2 X 8 array of randomly chosen letters is exposed for 50 millisec- 
onds. Then, after a variable delay, a black bar marker is presented for 
50 milliseconds either above one of the letter positions of the upper row 
or below one of the letter positions of the lower row. The subject’s task 
is to name the letter designated by the marker. A typical array and bar 
marker are shown in Fig. 1. A black circle that is used as a marker in a 


second experiment to be described later is also shown. The subject, of 
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LFTJMYNV 


Fig. 1 — Typical array of letters, bar marker and circle indicator. 


course, never knows before a given trial which letters will appear in the 
array, and which of the 16 letter locations will be called for by the 
marker. Thus, in order to perform well, he is required to store the array 
until the appearance of the marker. The sequence is illustrated in Fig. 2. 

A uniform field of 70 foot-lamberts was maintained constantly 
throughout the experiment, and the letters and marker appeared black 
against this background. This test field subtended a visual angle of 4 
degrees vertically by 5 degrees horizontally at the viewing distance of 5 
feet. It had a small dark fixation point in the center. Surrounding this 
field was a larger field, 12 degrees on a side, having a luminance of 30 
foot-lamberts. Each letter subtended one degree vertically by one-half 
degree horizontally. The black of the letters had a brightness of less 
than one foot-lambert. 


2.1 Procedure 


At the beginning of each session subjects were given two or three 
minutes to adapt to the bright screen. They were then given two warm-up 
trials with arrays that were not used in the experiment proper. On each 
trial the subject was given a ready signal and enough time to fix his eyes 
on the fixation point at the center of the screen. When fixated, he would 
signal the experimenter and the array and marker were then exposed. 
The subject was given as much time as he needed to make his response, 
but was encouraged to use his first guess if he was in doubt. He was 
given the correct answer after each trial. 

During each session 128 array-marker pairs were exposed, covering 
each of the 16 positions at each of 8 time intervals between array 
and marker. The same order of presentation of the 128 arrays was used 
in each session of the experiment, but the time interval and marker posi- 
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Fig. 2 — Sequence presented in a typical trial. 


tions were varied randomly, with the restriction that successive groups 
of 16 arrays each contained a marker in each of the 16 positions. Three 
sessions were run with each of the three subjects. 


2.2 Apparatus 


The tachistoscope used is that designed by Nielsen,’ which uses multi- 
channel television generating equipment and a set of gates controlled 
by timers for presenting a sequential display of three pictures on a single 
picture monitor. Each picture can be displayed for a preset time inter- 
val of N./60 seconds, where N is a number of television fields from 1 to 
99. Since all parts of a picture are not exposed simultaneously and a 
particular point on the monitor is illuminated for only 20 microseconds, 
exposure times are taken from the time a particular point is first seanned 
to the time the same point is scanned in a new picture. An exposure of 
50 milliseconds, preceded and followed by a white field, is illustrated in 
Fig. 3, which also shows the brightness of a point near one of the letters. 
Interlace is ignored since the center-to-center separation of the scanning 
lines subtended less than one-half minute of are at the observer’s eye. 

The time between the onset of an array and the onset of a marker is 
never an integral number of fields, owing to the fact that the marker 
does not appear in the same part of the vertical scan as the letter it 
designates. This small error has been ignored, since it is only —3 milli- 
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Fig. 3 — Brightness of letter and surround for exposure of 50 milliseconds pre- 
ceded and followed by a white field. 


seconds if the marker falls above the array and +3 milliseconds if the 
marker falls below. 


2.3 Results 


The results for the three subjects are shown in Fig. 4. The abscissa 
is the time in milliseconds between the onsets of array and marker. The 
ordinate is the per cent correct, corrected for chance on the assumption 
that the subject perceives correctly a certain percentage of the time 
P,, and guesses randomly from the 26 possible letters when he does not 
perceive correctly. On this assumption, the measured per cent correct 
Py = P, + (i — P,) (1/26), which yields the plotted P,,’s. Estimates 
of the standard error of these points, which are a function of the num- 
ber of trials (48) and the per cent correct, range from 0.07 at 50 per 
cent to 0.06 at 20 and 80 per cent correct. 

The vertical lines through zero and 50 milliseconds represent the 
onset and offset of the arrays. Negative time means that the marker 
came before the array. The point at zero time was taken after the rest 
of the experiment was completed because it required modification of the 
apparatus. 


2.4 Discussion 


Although it might be assumed that this experiment yields a reason- 
ably good description of the time-decay of the short-term visual storage, 
the curves obtained cannot be said to represent this decay for two rea- 
sons. First, the true storage would be expected to decay to zero for long 
enough time intervals. But these results decay to a final level of 
about 35 per cent for two of the subjects and 25 per cent for the third. 
Second, because the process of detecting a marker and reading a letter 
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Fig. 4 Results of memory experiment; “R.P.,”’ “V.V.”’ and ““G.M.” identify 
the three subjects participating. 


undoubtedly takes time, the measured performance suggests a storage 
time that is shorter than the true storage time. 

The fact that the measured decay curves do not fall to zero suggests 
that the measured performance contains components of a more per- 
manent memory, as well as the short-term memory component that we 
would like to measure. In this context, the 25 to 35 per cent final per- 
formance level (which represents 4 to 5.6 letters) is attributed to what 
the subject has read into his more permanent memory. 

Maximum performance measured when the marker preceded the 
array is 65 to 80 per cent. It is obvious, of course, that, if the marker 
preceded the array by a long enough time, performance would reach 100 
per cent. Why, then, doesn’t performance reach 100 per cent? The rea- 
son is not that some letters are outside the fovea, since individual letters 
exposed in any of the 16 positions of the array are clearly legible. The 
explanation seems to lie, rather, in the fact that letters in some positions, 
although perfectly legible by themselves, are not legible in the context 
of the array. This finding is illustrated by the plot of performance as a 
function of position shown in Fig. 5. The numbers | to 8 represent, from 
left to right, the positions on the upper line of the array, and 9 to 16 
the positions on the lower line. The percentage is based on the pooled 
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Fig. 5 Performance by position in array, three-subject average. 
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data of the three subjects, each point being based on 72 trials taken 
across all time intervals. All subjects show the same distribution, in 
which performance is better at the center and ends, and poorer in be- 
tween. Performance on the upper line is consistently better than per- 
formance on the lower. 


2.5 Summary 


In summary, the following can be stated: 

1. The visual system can store information for longer than 130 to 200 
milliseconds. 

2. This storage can be tapped selectively on a signal given by the 
experimenter. 

3. Resolution of the storage — or ease of reading-out — is disturbed 
when too much data is put in. Sixteen letters in a 2 X 8 matrix is enough 
to demonstrate this effect. 

4. This disturbance does not affect all items of such a stored array 
equally. It disturbs the center and end items least and those in between 
most. 

5. As an exercise, we estimated the amount of information in the 
store when the bar marker follows immediately after the array, and ob- 
tained the figure of 37 bits for the poorest subject and 54 bits for the 
other two subjects. 


Ill. ERASURE 


If persistence were the only property of the visual storage, it would 
be difficult to understand how we see at all in our normal, continually 


changing environment. A storage process normally also involves erasure, 


to assure that old information is out of the store before new information 
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is put in. Otherwise, new information and old would be inextricably 
merged in the store. The experiment to be described deals with the 
erasure properties of the visual storage. 

The procedure in this erasing experiment was almost identical to that 
of the memory experiment just described. The same subjects were used, 
and the same arrays of letters were presented in the same sequence. The 
essential difference between the two experiments was in the form of the 
marker used. In the first experiment the marker consisted of a vertical 
bar pointing to the letter; in this experiment it consisted of a black circle 
surrounding the letter, as was illustrated in Fig. 1. Such a circle pro- 
duces a curious effect upon the letter if the time delay between array 
and circle is chosen properly. This effect we call erasure. 


3.1 Results 


Fig. 6 permits a comparison of the performance by each of our sub- 
jects in the bar marker experiment with his performance in the circle 
experiment. The curves of all three subjects start at a relatively high 
level, ranging from 70 to 80 per cent, drop sharply to a minimum, rang- 
ing from 10 to 20 per cent and rise slowly to an intermediate level of 
25 to 40 per cent. 

When the circle precedes the array or follows immediately after, per- 
formance in the erasure experiment is not greatly different from per- 
formance in the first experiment. However, when the circle is delayed 
by 100 milliseconds, the difference between the curves is quite large. 
Then, with still longer time delays, performance in the circle experiment 
rises slowly until it reaches approximately the values obtained in the 
first experiment. Thus, the curves begin together and end together, 
with performance in the ‘circle’ 
tween. 


experiment significantly poorer be- 


3.2 Discussion 


This experiment shows how a later visual stimulus can drastically 
affect. the perception of an earlier one. This backward-in-time action of 
the circle implies that the first stimulus is delayed with respect to the 
second, or, more precisely, that the first stimulus is stored. The process 
involves more than simple delay, since, as shown by the first experiment, 
the subject has access to the information during the delay. 

The question arises as to why the circle has such a damaging effect 
on the letters and why the bar does not. The answer lies in their relative 
distance from the letter. In preliminary experiments it was found that, 
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when the bar was close to the letters, those parts of the letter near the 
bar were strongly disturbed. The bar was therefore placed far enough 
away from the letters to avoid this effect. All parts of the circle, however, 
are close to the letter. 

One can conceive of the observed action of the circle on a preceding 
letter in many different ways. It is possible, for example, that the main 
effect on the letter is a “stopping in time’”’, i.e., a quick substitution of 
the circle for the stored letter. Such a process could function as an erasure 
mechanism, since it would assure that new and old information are not 
confused in the store. It is conceivable, on the other hand, that the effect 
of the circle on the letter is of a different kind. Perhaps the disturbance 
is a result of the kind of mixing or averaging process that an erasure 
mechanism seeks to avoid, or perhaps the effect of the circle is primarily 
to reduce the brightness or contrast of the letter. 

We are inclined to reject the latter alternatives. The observed effect 
is clearly not due to averaging in time, for if the circle and the letter it 
surrounds are presented simultaneously, the legibility of the letter is 
hardly affected at all. Yet this is just the condition for which averaging 
should produce the most damaging effect. With regard to the possibility 
that the circle affects the brightness or contrast of the letters we can say 
nothing conclusive. Introspectively, however, a change in brightness or 
contrast does not appear to be the primary effect. 

The view that the second stimulus limits the time available for read- 
ing-out is more attractive than the other possibilities for several reasons. 
lirst, the rise found in the erasure curve with increasing delay of the 
circle after 100 milliseconds is consistent with the idea that increased 
delay of the circle allows more time for readout. A second reason for 
believing that the effect of the circle is primarily to limit readout time 
stems from our observation that the lowered performance obtained 
when the circle follows the letter is not independent of the number of 


letters involved. If a single letter is exposed in any one of the sixteen 


positions of the array and a circle is presented with a 100-millisecond 
delay — this is the delay that yields the poorest performance on our 
curves — this letter will be read correctly by our subjects 100 per cent 
of the time. On the other hand, if we expose four letters — which can 
ordinarily be reported perfectly —- followed by the circle with this same 
100-millisecond delay, performance is disturbed dramatically. Thus, we 
believe that the effectiveness of the circle in disturbing performance is 
related to how much reading has to be done before the circle appears. 
If only one letter must be read, the circle does not affect performance 
measurably because, according to this interpretation, the subject has 
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enough time to read the letter before the circle appears; if many letters 
must be read, the subject will not have enough time to read all of the 
letters before the circle appears. It is then likely that the circle will erase 
the letter it follows before it is read. Our final reason for thinking that 
the circle limits the time available is an introspective one. We find that 
a briefly exposed letter followed by a circle — even when it is seen as 
perfectly as it is when a single letter is exposed —— seems to persist for a 
much shorter time than it does when it is not followed by a circle. 

We are therefore inclined to say that the effect of the circle is to re- 
move previously stored information. On this interpretation, the observed 
increase in performance with increasing delay is attributed, not to loss 
of erasing effectiveness of the circle, but to the increased time available 
for readout. 

In the light of the above, the shape of the erasure curve may be inter- 
preted as follows: 

1. High performance when the circle follows immediately after the 
array is due to simple temporal averaging in the visual system. This 
results in array and circle being effectively superimposed, which does 
not significantly affect legibility. 

2. Decreased performance at slightly longer delays can be attributed 
to the change from the superposition condition to the erasure condition. 

3. The slow rise from the minimum with further increases in delay of 
the circle is attributable to the increased time available for the subject 
to read the letter before it is erased. At still longer times, when perform- 
ance is about the same as in the bar experiment, the circle no longer 
erases but acts as a marker. This suggests, as outlined in the discussion 
of the first experiment, that performance at times longer than 200 milli- 
seconds depends not on the contents of the short-term storage at that 
time, but on the number of letters that had been read into the permanent 
memory before that time. 

The suggestion that two closely timed stimuli are perceived as being 
superposed is testable. In what was essentially a repeat of the experiment 
just described, we substituted a circle filled with grid lines for the un- 
filled erasing circle. When simple superposition holds, such an ‘‘eraser”’ 
should be much more interfering than a simple circle. The results are 
shown in Fig. 7. 

The dashed lines are plots of the results obtained using an unfilled 
circle. The solid lines give the results obtained with the filled circle. It 
is seen that for delays longer than 100 milliseconds the difference be- 
tween the two curves is small; while for delays of less than 100 milli- 
seconds, the difference is large. This confirms that the disturbing effect 
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of a later stimulus on a preceding one may be of at least two varieties: 
the superposition type, which is understandable in terms of averaging; 
and the other effect, which we have called erasure. 

It should be mentioned that preliminary work has been done in which 
an array of eight letters is presented to one eye and the circle delivered 
to the other. It is found that erasure occurs under these conditions al- 
though it has not yet been determined how this binocular erasure com- 
pares with monocular erasure. 


IV. READOUT 


As we have already pointed out in discussing the bar-marker experi- 


ment, the process of detecting the presence of a marker and reading the 
marked letter undoubtedly takes some time. If the time required for 
this process could be measured, we would be able to correct the decay 
characteristic obtained in the bar marker experiment for this time and 
have a more accurate idea of the duration of the storage. A method for 
measuring this time is available provided that our conclusions about the 
action of the circle in erasing a letter are correct. Suppose we present 
simultaneously an array and a bar marker pointing to one of the letters 
in the array. Then, a short time afterwards, suppose we present an 
erasing circle around the marked letter. If the circle indeed removes the 
marked letter from the subject’s storage, his performance under these 
conditions will measure how well he can det ct the marker and read the 
letter when given only the time interval between the onset of the array 
and marker, and the onset of the circle. Such an experiment was per- 
formed, in fact, using the same subjects and experimental conditions as 
before. 


4.1 Results 


The results appear in Fig. 8. The abscissa is the time between the 
onset of the array—bar-marker combination and the onset of the circle. 
The three curves are similar in form. It is seen that when the circle 
follows by more than 100 milliseconds performance rises rapidly as a func- 
tion of the time between array and circle. This is true up to 200 milli- 
seconds for subject “V.V.” and 270 milliseconds for the other two sub- 
jects, later presentations seeming to have no further effect. Performance 
when the circle follows by less than 100 milliseconds is very much like 
that obtained in the erasure experiment using a circle without a bar 
marker. 
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4.2 Discussion 


Results of this experiment indicate that it takes a significant time for 
subjects to detect a marker and read the designated letter, the level of 
performance being a function of the time available for detection and 
reading. Maximum performance requires times of the order of 200 to 
270 milliseconds. Thus, the decay curves from the first experiment. in- 
corporate two effects: (a) storage time and (b) readout time. As we shall 
see in the next section, it is possible by means of the readout time meas- 
urement to correct for the latter factor and solve for the storage time 
alone. 


V. STORAGE TIME 


We indicated in Section III that performance in the bar-marker ex- 
periment is probably the result of two different types of performance 


on the part of the subject. First, there is a nonselective readout, which 
is independent of appearance of the marker; second, there is a selective 
process, which occurs only after the marker appears, when the subject 
has been cued to direct his attention to the single desired letter. The 
nonselective process is indicated by the finding that the subject’s per- 
formance does not fall to zero even when the bar marker appears at 
relatively long times (450 milliseconds) after appearance of the array, 
at times when, presumably, the short-term storage has already decayed. 
It was separated out and measured in the erasure experiment, in which 
it is apparent that, if the subject has not read the designated letter from 
his short-term storage before the circle appears, he cannot read it later 
because the letter is erased by the circle. 

We have no direct measure of the selective readout component. The 
effect of this component can be derived from the original bar-marker 
curves by subtracting out the nonselective component from the whole. 
Fig. 9 shows the result obtained by subtracting percentages obtained 
in the erasure experiment from those obtained in the bar-marker experi- 
ment. The subtraction is not a simple algebraic one. It is clear that if 
the subject reads out the correct letter by chance before the marker ap- 
pears, designation by the marker cannot improve his performance. It 
therefore seems reasonable to treat the probabilities of reading letters 
before and after appearance of the marker as independent. 

Designating these probabilities as Pz, and P, respectively, the total 
probability of reading the letter is Pr = Pg + (1 — Pe)P4. Pr is the 
per cent correct in the first experiment, P, that in the erasing experiment 
and P, , the per cent in Fig. 9, is calculated from P, = Pr — Ps/(1 — 
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Fig. 9 — Derived ‘‘selective readout”? performance curves. 


P,). The initial drop in the erasure curve is ignored because, as shown, 
it involves superposition and not simple erasure. 
The derived curves for subjects “V.V.” and “R.P.” are quite similar. 


They start at their maxima and drop to zero as would be expected. That 
of subject “G.M.” has the peculiar shape it does because the slope of 
his circle erasure curve between zero and 100 milliseconds is indetermi- 


nate. If this slope is estimated from the “filled circle” erasure curve, the 
the characteristic shown in Fig. 10 is obtained. This decay is quite similar 
to that of the other subjects. 

Using these derived curves, it is possible to estimate the duration of 
the short term visual storage. Note that the curves in Figs. 9 and 10 
represent that component of the subject’s performance accomplished 
after appearance of the marker. We will assume that this component of 
performance is limited by the time available to detect the marker and 
read the letter before decay of the storage. We have already determined 
experimentally (see Fig. 8) the times required to detect a marker and 
read a letter to various levels of performance. By adding these readout 
times at each level of performance to the appropriate times in Figs. 9 
and 10, estimates of the storage time are obtained. This process and the 
result are illustrated in Fig. 11. 

The solid lines represent the selective readout components taken from 
Fig. 9 and, for subject “G.M.,” from Fig. 10. The empty points were 
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Fig. 10 — “Selective readout’? performance curve for subject ‘‘G.M.”’ derived 
from grid experiment. 


obtained by adding times at various levels of performance taken from 
Fig. 8 to the points at the same levels of performance on the solid curve. 
Each point is therefore an independent estimate of storage time. It is 
seen that these points approximate vertical lines surprisingly well. 
The estimated storage times are 300 milliseconds. for ‘“R.P.”’ and 250 
milliseconds for the other two subjects. 


VI. CONCLUSIONS 


In the light of the experiments reported here, the following interpreta- 
tion seems plausible: The visual process involves a buffer storage whose 
read-in is very fast and readout relatively slow. The storage includes an 
erasure mechanism whereby new information put into the storage tends 
to erase what was previously there. This erasure is local in character, 
since erasure of a given detail depends on its distance from the areas 
where new detail is being put in. The storage time is of the order of 
one-quarter second. The storage capacity is more difficult to assess. A 
lower bound on the storage capacity, computed from performance ob- 
tained when bar marker follows immediately after array, yields a figure 


of 37 bits for the poorest subject and 54 bits for the other two subjects. 


This figure seems quite high, considering that the letter arrangement 
used in the experiment and the sharpness and contrast of the letters 
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were not such as to maximize performance. The highest figure, obtained 


in span-of-perception experiments is 25.36 bits. Sperling,’ using a tech- 
nique similar to ours, also not involving complete report, obtained a 
value of 64 bits. 

The interpretation above, of course, is a tentative one. These experi- 
ments have raised many more questions about short-term storage than 
they have answered. Particularly compelling are the questions of how 
the storage is scanned, and the stimulus factors involved in erasure. It 
is hoped that further application of the techniques used here will shed 
more light on these matters. 
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Synthesis of N-Port Active RC Networks 


By I. W. SANDBERG 


(Manuscript received August 16, 1960) 


The following basic theorem concerning active RC networks is proved: 

Theorem: An arlitrary N X N matrix of real rational functions in 
the complex-frequency variable (a) can be realized as the short-circuit ad- 
mittance matrix of a transformerless active RC N-port network containing 
N real-coeffictent controlled sources, and (b) cannot, in general, be realized 
as the short-circuit admittance matrix of an active RC network containing 
less than N controlled sources. 


I. INTRODUCTION 


It is often desirable to avoid the use of magnetic elements in synthesis 
procedures, since resistors and capacitors are more nearly ideal elements 
and are usually cheaper, lighter and smaller. This is especially true in 
control systems in which, typically, exacting performance is required 
at very low frequencies. The rapid development of the transistor has 
provided the network synthesist with an efficient low-cost active ele- 
ment and has stimulated considerable interest in active RC network 
theory during the past decade. 

Several techniques have been proposed for the active RC realization 
of transfer and driving-point functions.” It has been established that 
any real rational fraction can be realized as the transfer or driving- 
point function of a transformerless active RC network containing one 
active element. In particular, Linvill’s technique’ has been the basis 
for much of the later work. 

tecently, Sipress” has shown that any two of the four short-circuit 
admittance parameters of a two-port network can be chosen arbitrarily 
and realized with a structure requiring only one active element. It fol- 
lows that all four parameters can be realized with three active elements. 

The problem of determining the minimum number of controlled 


sources required to realize all N“ parameters of an arbitrary N-port 


immittance matrix is of considerable theoretical importance and has 
been of interest to network theorists for several years. The solution to 


329 





330 THE BELL SYSTEM TECHNICAL JOURNAL, JANUARY 1961 


this problem is stated in the abstract; its proof is the subject of this 
paper. 

In Section II we derive some fundamental properties of N-port net- 
works containing less than N controlled sources. The results are 
formulated in terms of inequalities involving the ranks of certain 
matrices. It follows from this study that at least N controlled sources 
are required for the realization of an arbitrary N X N immittance 
matrix. In Section III we make use of our previous results to establish 
an approach to the realization problem. This approach leads to a con- 
structive proof that N controlled sources are in fact sufficient. A nu- 
merical example illustrating the essential points in the synthesis tech- 
nique is presented in the Appendix. 


Il. N-PORT NETWORKS CONTAINING CONTROLLED SOURCES 


A controlled source is ordinarily understood to be an ideal two-port 
network-representation of a single branch-branch constraint. The four 
types of elementary controlled sources are shown in Fig. 1. Note that 
the two “hybrid sources”’ [Fig. 1(a) and (b)] form a complete set, since 
they can be appropriately connected in cascade to realize each of the 
other two. 

For our purposes it is convenient to generalize the definition of a 
controlled source to refer to any voltage or current source whose value 
is a weighted sum of certain prescribed voltages and currents. Specifi- 
cally, if the value of a controlled voltage or current source is denoted 
by Ap 


dp = Di Cpibs, (1) 
i=] 


where b, , bs ,-- -, bj are controlling currents and bj4; , bj42 ,-+-, bj4e are 


controlling voltages. It is assumed that the a, , c,; and b; are Laplace- 


- The four elementary controlled sources. 
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transformed quantities and that the c,; are real rational functions of 
the complex frequency variable s. 


2.1 The Short-Circuit Admittance Matrix of an N-Port Network Contain- 
ing Controlled Sources 


Consider the evaluation of the short-circuit admittance matrix of 
an N-port network containing a controlled source subnetwork as shown 
in Fig. 2. Denote by E and I respectively the column matrices of voltages 
and currents at the N accessible ports: 














Ty] 


i 


Let A be the column matrix of all / controlled current sources and m 
controlled voltage sources, and let B be the column matrix of all j cur- 
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N-port network containing a controlled-source subnetwork. 
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rents and k voltages influencing the controlled sources: 


ay, I; 
, 


ag I. 
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I; 
72 
ky 














7 
= E k -_ 


The relationship between A and B is assumed to be given by 
A = CB, (5) 
where C is a (1 + m) X (j + k) matrix of real rational functions in 
the complex frequency variable. 
With E and A treated as independent variables, we apply the super- 
position theorem to obtain 
Y,E + DA, (6) 
where Y) and D are defined by the equation. In particular, Yo is the 
N X N short-circuit admittance matrix of the N-port network with 
the value of all controlled sources set equal to zero. 
Similarly, we can express B as 


B = FE + GA, 


where the matrices F and G are defined by the equation. From (5) 


and (7), 


A = [U — CG] 'CFE, (8) 
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where U is the identity matrix of order 1 + m. Using (6), 
[Y — Y,] = D[U — CG]'CF, (9) 


where Y and Y) are the short-circuit admittance matrices of the N-port 
network with all controlled sources respectively operative and set equal 
to zero. In certain degenerate cases, Yo and/or the right-hand side of 
(9) will not exist. In such instances the network can be treated as a 
limiting case of a structure for which this difficulty does not occur. 


2.2 The Rank of [Y — Yj] 


Consider the maximum rank of the NV X N matrix [Y — Y)]. Since 
the rank of a matrix product cannot exceed the rank of any of its 
. 19 
constituent factors, 


rank [Y — Yo| S rank [C] = R,. (10) 


The elements of [Y — Yo] are real rational functions in the complex 
frequency variable. Assuming that this matrix has finite poles at s = 
81, 82,°°°, Sm Of multiplicity n;, m2,---,mm respectively, it can be ex- 
pressed as 


m n 
; l 


Y-Yl= DAs + DDB, 


(11) 


(1) 
k=0 k=1 (s — 8,)* 


where the A, and B°Y are coefficient matrices and in particular, the 
B‘) are residue matrices. 

From (11), the matrix of coefficients of the first term in the Laurent 
expansion at the pole s = s; is 


Bi? = (s — 8:)"{¥ — Yo] | ome; - (12) 

In view of (10), we have 
rank [B “| < R.. (13) 
Similarly, the leading coefficient of the matric polynomial in (11) is 


given by 


A, = lim + [¥ — Yul, 
and hence 


rank [A,] S R.. (15) 


Consequently, when R, < N, all k-rowed minors of the matrices 


B’), and A, vanish, where k = R, + 1, R, + 2,---,N. 
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Inequalities (13) and (15) shed considerable light on the fundamental 
properties of an N-port network containing a controlled source sub- 
network. In fact, at poles of Y which are not poles of Yo , these condi- 
tions yield explicit restrictions on the Y matrix. For example, let Y be 
the admittance matrix of an active RC network and take Yo to be the 
corresponding passive RC matrix obtained from Y by setting all con- 
trolled source coefficients equal to zero. It is well known that Yo must 
be regular everywhere in the complex plane except at infinity and at 
points on the negative-real axis where only simple poles may occur. 
Hence Y, cannot influence the coefficient matrices B&?, at any mul- 
tiple-order pole or at any pole not on the negative-real axis. In par- 
ticular, the rank of the residue matrix at any simple complex pole can- 
not exceed R, , the rank of the matrix C. 

The rank of C, of course, cannot exceed the number of its rows or 
columns, whichever is smaller. That is, 


R. <= min [j + kil + ml. (16) 


This means that R, cannot exceed the number of controlled sources or 
the total number of controlling voltages and currents, whichever is 


. ‘2 . . (l) . 
smaller. Consequently, if any of the prescribed B_,, are to have full 


rank at a pole of Y which is not a pole of Yo, the controlled source 
subnetwork must include at least NV controlled sources and at least N 
distinct control ports. T 

A similar development, of course, can be carried out in terms of the 
open-circuit impedance matrices Z and Zy. Note that these results are 
valid for controlled source coefficients c,; which may be any set of real 
rational functions in the complex-frequency variable. Note also that a 
driving-point immittance can be regarded as a controlled source, since 
such immittances impose a constraint which is merely a special case 
of (1). 


Ill. N-PORT ACTIVE RC REALIZATION 


We begin the study of the N-port realization problem by considering 
an active RC network containing one controlled source. Specifically, 
consider an (N + 2)-port passive RC network characterized by the 
(N + 2) X (N + 2) short-circuit admittance matrix Y, and suppose 


t The realization of an arbitrary NX N matrix of constants as the short-circuit 
admittance matrix of an N-port network containing positive resistors, ideal trans 
formers and controlled sources also requires, in general, at least N controlled 
sources. This follows from the fact that, in this case, Y, is the matrix of a non 
negative quadratic form, and hence it is possible to prescribe constant matrices 
Y such that, for the entire class of matrices Yo ,{Y — Yo] is of rank N. 
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Fig. 3 — Active RC network containing one controlled souree—canonical 
subnetwork. 
that a current-controlled voltage source is connected between ports 
N+ 1andN + 2 as shown in Fig. 3. Denote by Yo and Y the VN K N 
short-circuit admittance matrices relating the column vectors 
T By Ti 
De I, 














| Ey | Ty | 


when the controlled source coefficient respectively vanishes and is equal 
to R. 


The matrix Y is given, as a special case of (9), by 


Yi,n42 


R , r . 
= i+ ame ° (Owais Gwainl, 


Yn wie 


where Y, is the matrix of elements in the first N rows and columns of 


Y. It is convenient to express (18) as 


Pi.w42 
Y Y k ° lp D } 19) 
oh Bp ake q(q + Rpwsi.w+2) : Py+ia'**Pweiwiy (hs 


Py.n+2 
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where g and the pj, are polynomials, and 


iin = Pi. (20) 
q 
It is evident from (18) or (19) that, as anticipated, [Y — Yo] has 
unit rank. 


3.1 N-Port Synthesis 


Our objective is to obtain an expression involving Y similar to (9) 
with a right-hand side of rank NV. We know that a network character- 
ized by such a relationship will require at least N controlled sources. 

It is well known that a rank N matrix can be expressed as a sum of 
N rank | matrices.” This suggests that the realization of Y can be ac- 
complished with N networks connected in parallel. We shall specifically 
consider the parallel connection of N networks of the type shown in 
Fig. 3. 

Assuming that the scalar coefficient on the right-hand side of (19) 
is the same function of s for each of the N subnetworks, we obtaint 


~(2) 


N R 
Y- DY = - * 
ing qq + Rpwai.ws: 


where 


(23) 


The sum of matrix products in (21) can be written as a single matrix 
with the element in the jth row and kth column given by 


N 
>. So rieies. (24) 


i=! 


This matrix can therefore be written as the product of the following 
two matrices: 
+t The networks are assumed to be such that admittance matrices add without 


the use of ideal transformers. This is justified later by employing balanced struc- 
tures. 
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~(N) ~(1) ~(1) 
* P1,N+2 Py+i, 1°° *Pn+1,N 


P,P, ~ : 

~(N) ~(N) 
PNn+1,1° . *PN+1,N 
From (21) and (25), 


R 
Y - Yo = = -~ P,P: , 
4 q(q + Rivaixys) 


where 


Yor . >> Yo: ° 


Let the prescribed short-circuit admittance matrix Y be given as 


where D is the common denominator polynomial of the elements in Y 
and |N,;] is a matrix of iis ila Similarly, write Yor as 


Yor = ; [pj] = l 5 [p\) | 


q k=1 


From (26), (27) and (28), 


R 
Dpi; — Ni — P,P,. 
5 Pp q | q(q + cand iP 


In (29) let terms be identified as follows: 


" l 
Pyu.n+2 = R (DP — @), 


PP, = 1 1Dpy ~ aN uh 
\ 


At this point we have reduced the synthesis of the N-port admittance 
matrix Y to the determination of N realizable (NV + 2) X (NV + 2) RC 
network matrices Y‘" whose elements satisfy (30) and (31). 


3.2 Sufficient Conditions for the Realization of Y 
The matrices Y°" can be expressed as 
des ¢ 


«ad +8.° %+ 22° ——. 
jal 8 + 9; 





338 THE BELL SYSTEM TECHNICAL JOURNAL, JANUARY 196] 


where ‘“‘deg q’’ means the degree of the polynomial q, and where the 
a; are real and satisfy 


OS ay eet < Gdeg q- 


If the coefficient matrices K, , Ky and K; are ‘‘dominant-diagonal”’ 
matrices,t (32) can be realized as a transformerless balanced RC(N + 
2)-port network.” 

Assume that Yor has been chosen so that 

(a) its coefficient matrices satisfy the dominant-diagonal condition 
with the inequality sign; 

(b) the matrix (1/R){Dp;; — qN,;| can be expressed as the product 
of two polynomial matrices P; and P, with the property that (1/q)P, 
and (1/q)P. are matrices of realizable RC transfer admittances (these 
admittances are assumed to have poles at infinity only when Yor has a 
pole at infinity); and 

(c) the function py4i.v42 satisfies the realizability and regularity 
constraints stated in (b). 

If (a) is satisfied, we can write Yor as the sum of N matrices Yo, , 
each of which has coefficient matrices that satisfy the dominant-diag- 
onal condition with the inequality sign. Recall that Yo, is the matrix of 
elements in the first V rows and columns of the (NV + 2) K (N + 2) 
matrix ¥". To obtain Y‘", we border Y,; with two additional rows and 
columns of elements. All but three of the required numerator poly- 
nomials are determined by the entries in the polynomial matrices P, 
and P, which satisfy (b). Of the remaining three polynomials, py4i.w42 
is given by (30) and is assumed to satisfy (¢), while pysi.wy1 and 
Py+e.~4e may be chosen freely to assist realizability, since they are 
unrestricted by (30) or (31). 


The realizability of Y' can be ensured by having it exhibit the 


dominance characteristic, and this can always be done by choosing the 
scale factors of the polynomials pysi.vi1 and pyss,vs2 as well as the 
value of R, the controlled source coefficient, to be sufficiently large. 

Hence (a), (b) and (c) are sufficient for the realization of Y. To 
make further progress, we next establish conditions that permit P = 
(1/R)|Dp.; — qNj;\ to be written as the product of two matrices with 
polynomial elements of lower degree. 


+ A dominant-diagonal matrix M has elements mj, which satisfy 


mi; > 3s m jx | 


ky) 
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3.3 Factorization of the Matric Polynomial P 


Let L be the degree of the highest degree polynomial in P = (1/R)- 
[Dpi; — qNij|, and suppose that the zeros of 


NL 
P 
det P = >> us 
k=0 
include K distinct real zeros at s = s;, (7 = 1,2,---,N,---,K). 
Consider the result of determining a nonsingular matrix Q with real 
constant elements such that every element in the 7th column of PQ 
has a zero at s = s;, (7 = 1,2,---,N). If indeed this can be done, P 
can be written as 
1 1 “es 
P = (PQ)Q =P(DQ ), (33) 
where D is the diagonal matrix diag [s — s,,8 — 8 ,---,s — sy], and 


the degree of the highest degree polynomial in P’ is L — 1. This is 
equivalent to removing a linear factor of the matric polynomial P: 


Pp = > vA; = P vn, | DQ 


j=l j=l 


= P ’A'Q | QDQ " 


j=l 


L—1 
= P vas | (sU — B), 


j=1 


where U is the identity matrix of order N and 
B = Q diag [s: , 2 ,-+-, sv]Q '. (35) 


If (N — 1)L < K, a matrix Q having the required properties exists 
and can be constructed as follows. First, note that at any zero of det P, 
say at s = s,, the column rank of P is necessarily less than V, and hence 
there exists a relationship of the form 


N 


0 = > a;{P;(s:)], (36) 
j=1 
where [Pj(s,)] is the jth column vector of P evaluated at s = s;, and 
the a;, are not all zero. Note also that at no more than (N — 1)L of 
the zeros of det P is it possible to determine alphas, not all zero, which 
satisfy 
N 


0 = 5 a ;i[P j(s1)], 


jk 
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where k is any one of the integers [1,2,---, NV]. This follows at once from 
the fact that all nonidentically vanishing determinants, formed from 
det P by replacing the kth column of det P with a column of constants, 
vanish at most at (N — 1)Z points. Therefore, if (NV — 1)L < K, 
there must exist at least one equation of the type (36) for a real zero 
and with a,; * 0. In other words, there exists a nonsingular matrix of 
real elements 








QNnk 


such that every element in the Ath column of PQ, has a real zero at 
s = s,. Note that the elements in all columns except the kth remain 
unchanged. Hence the matrix Q can be constructed as a product of NV 
matrices Q; chosen so that every element in the 7th column of 


PI[Q,;, i = 1,2,--- 
j=] 


has a real zero at s = s;. 

To summarize, if (V — 1)L < K, N distinct real zeros of det P can 
be removed as a linear factor of the matric polynomial P. The remain- 
ing polynomial is of degree L — 1 and all coefficient matrices are real. t 

To simplify the discussion, we have not considered certain extensions 
of the factorization technique. It is possible, for example, to carry out a 
similar development with respect to the rows of P. This permits the 
removal of a linear factor that premultiplies the remaining matric poly- 
nomial. 


3.4 Consideration of Conditions (a), (b) and (c) 


The admittance matrix Yor can be made to have dominant-diagonal 
coefficient matrices by choosing any N X N realizable RC admittance 


t This implies that the matrie polynomial P can be written as 


P = CI] (sU — B;) 


i=] 


when det P has NLU distinct zeros. When these zeros are all real the coefficient 
matrices C and B; are also real 
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matrix, with elements of suitable degree as determined subsequently, 
and multiplying each diagonal entry by a sufficiently large positive 
real constant p. Hence condition (a) is easily satisfied. Denote the 
matrix determined in this way by 


, ~ 
pPu Piz as Pin 


ppr2 








| Pw 


The polynomial det P can be written as 


am Rees‘ oe fe ee 
det P = det R [Dp.; — qNij] = (*,) D 


where R(s)/p¥ is a polynomial with degree not exceeding NL and with 
all coefficients that approach zero as p approaches infinity. We shall 
assume that the degree of p;;, deg p;;, has been chosen to be inde- 
pendent of the index 7. Note that, as p approaches infinity, N deg pj; 
zeros of det P approach the zeros of 


ad , 
I] Di 


The zeros of this product can be chosen to be distinct and different from 
those of D. Hence, for a sufficiently large value of p, (a) is satisfied and 
det P has at least N deg p;; distinct real zeros. 
Next, consider condition (b). The degree of the highest degree 
polynomial in P is given by 
L max [max deg p;; + deg D, max deg N;; + deg q| 
= max [deg p;; + deg D, max deg N;; + deg q]. 
Hence, 
L = deg pi; + max [max deg N;; — ¢, deg D] 
deg pi + Le, 
where 


deg pi: = deg q 
(43) 


deg p;; = deg q + 1. 


To remove k linear factors of the matric polynomial P as described 





>» 
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in Section 3.3, it is sufficient, after removal of the (k — 1)th factor, 
that 


(N — 1)[deg pii + Le — (k — 1)] < N deg pi: — N(K — 1). (44) 


If k = L, factors are removed, P could be written as the product of 
two matrices, one of degree L, and the other of degree deg p;; . Sub- 
stituting this value of k into (44) gives the required relationship be- 
tween L, and deg p,; : 


NL, — 1 < deg pis. (45) 


Hence conditions (a) and (b) are satisfied? with deg p;; = NL, . Finally, 
it is evident that condition (c) can be satisfied simultaneously, since 
Pwii,w42 can be chosen to have any degree not exceeding deg pi; . 

This proves the theorem stated in the abstract. 


IV. CONCLUSION 


We have proven that N is the sufficient and, in general, minimum 
number of controlled sources required to realize an arbitrary N X N 
matrix of real rational functions as a transformerless active RC N-port 
network. A canonical structure is a parallel combination of V networks, 
each containing a single controlled source. The type of controlled source 
employed is one of the two basic elementary controlled sources. Similar 
developments can be carried out for other types of controlled sources. 

Further work is indicated in several directions. It is desirable to avoid 
the use of balanced networks. A detailed investigation of matric poly- 
nomial factorization may shed some light on this possibility. A major 
difficulty stems from the fact that relatively little is known about the 
realization of transformerless passive RC networks. Even so, it is almost 
certain that more practical canonical structures will be discovered. 

It is noteworthy that the analytical machinery employed here pro- 
vides insight into other fundamental questions. For example, it is easy 
to show that all NV resistors in Oono’s passive N-port realization” are in 
fact necessary. Similarly all V-negative and N-positive resistors in Car- 


lin’s active N-port realization” are necessary. 
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+ This is a pessimistic statement of the required degree of p,; , and results from 
the particular technique employed to factor P. A more detailed study of matric 
polynomial factorization, as yet incomplete, indicates that the degree of p;; can 
generally be reduced by a factor of N. 
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factoring a matric polynomial presented in Section 3.3 is based on a 
suggestion by 8. Darlington. 


APPENDIX 
Synthesis of a Two-Port Network — A Numerical Example 
To illustrate the main points in the synthesis technique presented in 
Section III, we consider in detail the synthesis of a two-port network. 
This example demonstrates also that (45) is not a necessary condition. 
Let the prescribed 2 X 2 matrix be 
l 


Y = D [Nil 


I e+e? +243 
~~ F t+stilt®+s4+4 @t@4+s45]° 
We choose Yor as the following matrix that obviously satisfies the domi- 
nance condition with the inequality sign: 


] 
Yor = [pij| 
q 


1 5(s + 1)(s + 3) 0 
(s + 2)(s + 4) 0 5(s + 1)(s + 3)]° 


From (30), (31), (46) and (47), 


Lie * 
D3.4 = ~ R (os + 7), 


is‘ + 188° + 248° + 15s —1 —s* — 7s* — 17s” — 26s — 24 
—s'—7s° — 18s” — 32s — 32 4s‘ + 188° + 218° — 3s — 25 
Consider the factorization of P into two matric polynomials of the 
second degree. The factors of 
R™ det P = 15s° + 130s’ + 420s° + 555s° — 152s" 
— 1629s° — 2474s" — 1972s — 743, 
determined with a digital computer, are 
(s + 1.0707018)(s — 1.6223931)(s + 3.0014915)(s + 2.6871002) 


(50) 
-(s + 1.3191886 + 71.2215876) (s + 0.4456939 + 70.9460882 ). 
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Denote by 8; , sz, 8; and s respectively the zeros of the first four factors 
in (50). 

First, we determine a matrix Q,; such that both elements in the first 
column of PQ, have a zero at s = s,. Ats = 8s, 


an[Pi(s1)] + an[Po(s,)] = 0. (51) 
By evaluating the pair of polynomials in either row of (49) at s = s, we 
obtain: 
0.76249 ay + ay = 0. 
Hence, 
] 0 
Q: 


—0.76249 1 


From (49) and (52), 


l Bi 
PQ, = R [aj], (53) 


where 

ay = (4.7625s° + 18.2382s° + 17.4347s + 16.1575)(s + 1.0707), 
dx, = —(4.0499s° + 16.3886s° + 16.4651s + 12.0835)(s + 1.0707). 
ay. = —(s' + 7s + 178° + 26s + 24), 

doy = (48° + 188° + 21s" — 3s — 25). 


Next we find a matrix Q» such that the first column of PQ,Q, is iden- 
tical to that of PQ: , and both elements in the second column of PQ,Q. 


have a zero at s = s.. The evaluation of polynomials as before leads to 
1 0.48643 


Q, = ‘ ( 54) 
0 ] 


At this point, P can be expressed as 


l ; 
Pp = p oisl diag [s + 1.0707, s — 1.6223]Q"', 


where Q’* = (Q,Q.) *, and 
bu = 4.7625s° + 18.2382s° + 17.4347s + 16.1575, 
by = —(4.0499s° + 16.3886s° + 16.4651s + 12.0835), 
bis = 1.3166s° + 6.48818" + 11.5058s + 9.6064, 


bes = 2.0300s° + 11.21248" + 22.6465s + 19.2894. 
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A second linear factor of P can be removed by repeating this process. 
Specifically, if the zeros at s = s; and s = s, are removed respectively 
from the first and second columns of [b;;], P can be expressed as 


Pp = P,P, 


with 


4.3560s° +3.1605s+ 4.3961 1.31668’ +2.9503s+3.5781 


— (4.67668? +5.8136s+6.0077) 2.0300s°+5.7576s+7.1753 


1 [0.6292s°+2.5622s+2.0221 —0.4863s"— 1.9803s— 1.5629 


BR | 0.95688" +1.5419s — 2.7648 0.84998’ ++0.5007s — 4.7910 


where @ is an arbitrary nonzero real parameter. 

To determine Y" and Y, first write Yor as the sum of two matrices, 
Yo: and Yoo, that satisfy the dominance condition with the inequality 
sign. The following choice is clearly acceptable: 


You — Yoo — 5Yor i 


<yil) ey (2) ° 
Hence, Y'' and Y™ are given by 


lw (s) 


Pu 


0 








where 
~(2) 
a 


pi = piu’ = px’ = pra’ = $(s + 1)(s + 8). 


The polynomials p33’, ss’, pas’, and pis’ are unrestricted by (31) and 
hence, for simplicity, can be chosen to be 3(s + 1)(s + 3). The remain- 
ing polynomials are obtained from (25) with P; and P» given explicitly 
in (56). It is evident that finite, nonzero parameters 8 and R can be 
determined so that the matrices ¥’ and Y® satisfy the dominance con- 
dition. The realization of each of these matrices takes the form shown 
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Fig. 4 — Realization of Y or ¥® for two-port network example. 


in Fig. 4, where the rectangles enclose transformerless passive balanced 
1”) 


RC structures.” 
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